The move comes as part of a Palestinian drive to refer their demands for an independent state in Israeli-occupied territory to more international bodies since US-backed peace talks collapsed last month.
Malki said militant settlers known as the "Hilltop Youth" and vigilantes who use the "price tag" slogan "practice terror ... constantly against the Palestinian people, their land, holy places and property."
"Price tag" refers to retribution the settlers say they will exact for any attempt by the Israeli government to curb settlement in the West Bank, an area Palestinians seek as part of a future state.
"These groups play a role in killing, incitement to violence and spreading the culture of hatred and racism," he said in the letter sent to Russia, the United States, Canada, the European Union, the United Nations, the Arab League and the Organization of Islamic States.
Palestinian anger has mounted over incursions by militant Jewish settlers into Palestinian villages where they have torched buildings, scrawled hateful slogans and attacked residents.
"Price tag" attack on mosque in Fureidis (Photo: Hassan Shaalan)
The European Union and Western countries have for decades referred to several Palestinian armed groups as terrorist organizations and, more recently, have also included the Israeli ultra-nationalist militant group Kahane Chai.
A US State Department country report on terrorism published last month described the raids by militant settlers, but did not designate them as terrorist groups.
"Price tag" attack on mosque in Umm al-Fahm (Photo: George Ginzburg)
The UN Office of the Coordinator for Humanitarian Affairs in 2013 reported 399 settler attacks resulting in Palestinian injury or property damage.
Israeli officials did not immediately respond to requests for comment on the Palestinian move.
Since the UN recognized Palestine as a non-member state in 2012, Palestinians have stepped up a campaign to join UN bodies and submit their complaints to international opinion.
Israel sees these moves as unilateral and harmful to peace negotiations, which broke down in April amid mutual blame.
Israel's justice and internal security ministers asked the cabinet last week to introduce the "terrorist group" label for the attackers.
Despite dozens of arrests by Israeli security forces of suspects over the past year, there have been few convictions. Police say many are minors to whom courts show leniency.
The frequency of attacks has risen sharply over the past month since the Israeli military demolished structures in a West Bank settlement built without government authorization.