A total of 20 people lived in the four buildings, two of them located in the Issawiya neighborhood and two in Beit Hanina, occupants told AFP.
They had been served demolition orders because they did not have the necessary construction permits, police spokesperson Luba Samri told AFP.
She added the demolitions went ahead without incident.
In 2013, Israel destroyed 99 buildings in annexed east Jerusalem, leaving 298 people homeless, according to United Nations humanitarian affairs agency OCHA.
Palestinians and human rights groups in the city say Israel rarely grants the permits, forcing residents to build homes without them.
Israel defines the whole of Jerusalem as its "eternal and indivisible" capital, while the Palestinians claim the eastern sector, captured during the 1967 Six Day War and unilaterally annexed shortly after, as the capital of the state to which they aspire.