In the French capital, former presidents Francois Hollande and Nicolas Sarkozy joined a rally led by Prime Minister Edouard Philippe on Republic Plaza.
Political parties from across the spectrum participated in the nationwide rallies with the theme "That's enough", although Marine Le Pen's far-right National Rally party held a separate event.
French President Emmanuel Macron went to the Shoah Memorial, a Holocaust museum in Paris, to observe a moment of silence with parliament leaders.
"Every time a French person, because he or she is Jewish, is insulted, threatened -- or worse, injured or killed -- the whole Republic" is attacked, Macron said at a news conference in Paris.
Hours before the rallies started, Macron visited the vandalized Jewish cemetery in Quatzenheim, a small town in the northeastern Alsace region. He said he felt shame at the sight of the defaced grave markers.
"This looks like absurd stupidity," the French leader said, looking visibly sad and concerned.
Macron observed several moments of silence in front of the vandalized graves while local Jewish community representatives stood by.
"We will take action," he promised.
France is home to the world's largest Jewish population outside Israel and the United States. Among the incidents arousing worries of renewed anti-Semitism was a torrent of hate speech directed at Jewish philosopher Alain Finkielkraut during a Saturday march by yellow vest protesters.
In recent incidents, swastika graffiti was found on street portraits of Simone Veil -- a survivor of Nazi death camps and a European Parliament president who died in 2017. The word "Juden" was painted on the window of a bagel restaurant in Paris, and two trees planted at a memorial honoring a young Jewish man tortured to death in 2006 were vandalized, one cut down.
Two youths were arrested Friday after they allegedly fired shots at a synagogue with an air rifle in the Paris suburb of Sarcelles, where a large Jewish community lives. Sarcelles mayor Patrick Haddad told BFMTV on Tuesday that prosecutors believe the motive was anti-Semitism.
According to sociologist Danny Trom, author of "France Without Jews," thousands of Jewish people leave France every year because of anti-Semitism.
"This is a low-intensity war, perhaps, but let's not forget the murder of children killed at close range by Mohamed Merah in a school," Trom told French magazine Telerama, referring to the 2012 slayings of three children and a teacher from a Jewish school by an Islamic extremist in the southwestern city of Toulouse.
"It is without equivalent in the history of France," he said. "Jews have been present in France since the dawn of time. Now, the pressure is such that they are led to consider their country inhospitable."
The French government reported a big rise in anti-Semitism last year: 541 registered incidents, up 74 percent from 311 in 2017.
Leaders from France's main religious communities, including Christian, Muslim and Jewish representatives, met at France's Interior Ministry on Tuesday. In a joint declaration, they solemnly condemned anti-Semitic acts and called on people to make individual commitments to combat all forms of racism and hatred.