The damage occurred sometime after the cemetery was closed Monday evening and before it was re-opened Tuesday, and authorities are investigating, said Giorgio Colombo.
The vandalism - along with the burning of Israeli flags during a recent march to commemorate Italy's liberation from fascism - has raised concerns about anti-Semitism in Italy's financial capital.
"I believe that the Milanese will know how to react to this cowardly act," said Milan Deputy Mayor Riccardo De Corato, denouncing "Anti-Jewish vandalism."
'Anti-Semitism must be battled'
"It was a very grave act, without precedent," Said Yasha Reibman, spokesman for the tiny Jewish community in Milan.
"Whatever was behind it, we insist that anti-Semitism in every form must be battled," Reibman said in a telephone interview.
Milan's Jewish community numbers about 12,000 in the metropolis of 1.3 million.
Last month, Italian authorities opened a probe into anti-Israeli protests that flared on the sidelines of a march in Milan on April 25 to mark national Liberation Day.
The protests, in which Israeli flags were trampled and burned and slogans shouted in support of Palestinians, were apparently sparked
The Italian news agency ANSA said police said initial investigation indicated that the cemetery rampage appeared to be more vandalism than anti-Semitism. There was no graffiti on the stones nor was any claim of responsibility made, officials said.
"This does not lessen the sense of offense and the pain that we feel at seeing stones ripped out and destroyed," The news agency quoted Emanuele Fiano, a left-wing Milan politician and a Jewish community official, as saying.