Mayor Mauricio Macri told an Argentine news agency Palacios had resigned for "personal reasons".
Palacios' appointment to the position was criticized by families of victims of the attack, which claimed the lives of 85 and injured an additional 300.
In 2008 the 60-year old investigator was questioned on suspicion that he purposefully distorted the investigation of a Syrian businessman suspected of involvement in the attack.
Macri said the resignation was "a noble gesture that only lends him respect". He said Palacios had not wanted to cause the government any public embarrassment, as Macri had supported his nomination.
Palacios' critics also claimed he had taken part in the suppression of a protest against the government in December of 2001, in which 30 residents of the city were killed.
The Buenos Aires Jewish center attack made headlines once more last week, when Iran declared a man suspected of involvement in the attack was appointed Iran's new defense minister.
Ahmed Vahidi is wanted by Interpol for allegedly planning the attack and ordering it to be carried out. An official Iranian source has said that the world's anger over the nomination was one of the factors that caused President Mahmoud Ahmadinejad to approve it.
Argentina issued a condemning message following Vahidi's appointment, but Iran was dismissive and warned the country against becoming involved in its affairs.
AFP contributed to this report