Interior Minister Bernard Cazeneuve said in a statement Sunday that a special criminal investigation team is at the cemetery in Sarre-Union, near the German border, and authorities will do "everything" to pursue the vandals.
Meyer Habib, a French-Jewish parliamentarian, said "once again Jews are being targeted, yesterday it was Denmark, and today its graves in France. For the terrorist there is no difference between life and death – the important thing is harming Jews."
Habib praised the French government for working to curb such instances, which he said "have united the nation" but notes that "these types of incidents are getting worse and more concerning. We fear for our children."
French President Francois Hollande said in a statement that "France is determined to fight relentlessly against anti-Semitism and those who want to attack the nation's values."
Hollande visited the Danish Embassy in Paris on Sunday, and a crowd waving candles gathered to show solidarity with the victims of the Copenhagen attacks.
"We need stand together in Europe and in all the world wherever jihadis try to threaten democracy," said Sacha Reingewirtz, president of the Union of Jewish Students of France.
Jewish and Muslim gravesites and places of worship see sporadic but frequent vandalism in France. The latest incident comes amid religious tensions in Europe after Islamic radicals attacked a kosher market and a newspaper in Paris last month, and similar attacks in Denmark this weekend.
Cazeneuve urged calm and tolerance. Many French Jews feel increasingly worried about anti-Semitism after the attack on the Charlie Hebdo satirical magazine was followed by an attack on a Jewish supermarket in Paris that left four dead.
A record numbers of French Jews have already begun emigrating to Israel in wake rising anti-Semitism and a weak economy, peaking at 7,000 last year.
France's leading Muslim groups denounced the attacks in Paris and Copenhagen, and are frustrated that Islam is often associated with terrorism.
Following the deadly attack France's Prime Minister Manuel Valls said in an interveiw that an exodus of Jews from the country would mark "the failure of the French republic."
AP and AFP contributed to this report.