Levy, 72, is said to be unconscious and in serious condition.
The former military man had previously suffered a stroke and was hospitalized at the Sheba Medical Center at Tel Hashomer. Since then, he had been confined to a wheelchair but continued to be in the public eye even serving as an advisor to a large Israel transportation corporation.
Nicknamed 'Moishe and a half' because of his height, Levy was the first IDF chief of staff to begin his career in the IDF as opposed to as a member of pre-1948 Jewish militias.
Levy was born in Tel Aviv to Iraqi immigrants to Israel. He was drafted into the IDF in 1954, serving first in the Golani Brigade and later in the paratroopers.
Levy took part in the famed battle over the Mitla pass in the Sinai Peninsula during the 1956 Suez campaign.
The former chief of staff was appointed to the position by then-Defense Minister Moshe Arens.
During his tenure, Levy presided over the IDF withdrawal from Lebanon in 1985 and initiated the establishment of two new IDF brigades: the Nahal and Givati brigades.