Independent candidate Moses Elisaf won 50.3 percent of the vote in the municipal election in the city of Ioannina in northwestern Greece. He received 17,789 votes, 235 more than his runoff opponent.
He told the Associated Press that his election as one of Greece's 332 mayors shows "stereotypes and prejudices" are being overcome.
Greece has about 6,000 Jews in its population of 10.8 million. The country has seen a spate of anti-Semitic vandalism in recent years, coinciding with the rise of a Nazi-inspired party.
Ioannina had a relatively large Jewish community, dating back to the days of the destruction of the Second Temple in Jerusalem in 70 CE. Among the many Byzantine churches there is one synagogue which continues to hold religious services to its shrinking congregation of about 50.
There were around 2,000 Jews living in Ioannina on the eve of World War II, when almost all of the Jewish population was slain at the Auschwitz death camp in Poland in 1944.
"Elisaf's victory in Ioannina is of great significance and sends a strong message to the Jews in Greece," said Avi Azoulay, an Israeli resident of Thessaloniki, who is the former spokesperson for the Israeli town of Beit Shean.
"(There is) a feeling of pride among the Jews who still live in Greece and among the non-Jewish Greeks who are happy their country made history in the municipal elections," says Azoulay, who owns a company for Israeli tourism to Thessaloniki and northern Greece.
David Saltiel, the president of the Central Board of Jewish Communities in Greece, also hailed Elisaf's victory as a rejection of anti-Semitism.