The comments by the head of the CRIF Jewish umbrella group came just hours before President Francois Hollande was due to give a speech at its annual dinner, which typically draws leaders of all major religions as well as politicians.
They rekindled tensions in a country which is home to Europe's largest Jewish and Muslim minorities, weeks after political leaders called for a spirit of 'national unity' in response to a series of attacks by Islamist militants that killed 17 people and three attackers.
"We need to say things clearly: all violence today is committed by young Muslims," Roger Cukierman told Europe 1 radio. "Of course, it's a tiny minority of the Muslim community and Muslims are the first victims."
Cukierman had been asked whether the far-right National Front party bore any responsibility for anti-Semitic acts. Five minors were arrested last week for having damaged hundreds of tombs earlier this month in a Jewish cemetery in eastern France.
"I think that all of us in the Jewish world are aware that behind (National Front leader) Marine Le Pen - who is blameless on a personal level - there are all the Holocaust deniers," he said.
Muslim advocacy groups have reported a rise in Islamophobic acts, including vandalism on mosques, after the January attacks at the Charlie Hebdo satirical weekly and a kosher supermarket on the edge of Paris.
The CFCM Muslim umbrella group condemned Cukierman's comments in a statement as "irresponsible and unacceptable" and said it would not attend its dinner.
"I don't think Mr. Cukierman is someone who will contribute to us all living together peacefully," Abdallah Zekri, head of a CRIF department that tracks Islamophobic incidents, told BFMTV television.
Cukierman told French media later that he was disappointed that the CFCM would not be attending, though he did not address his earlier comments directly.