"To hear what Israeli officials say, one would think Israel was showering Lebanon with roses during its last aggression," Prime Minister Fuad Siniora said of the 2006 summer war between Israel and Hizbullah, in which over 1,200 Lebanese - mostly civilians - were killed.
Olmert said Israel did not use all means to respond then, but "if Lebanon becomes a Hizbullah state, then we won't have any restrictions in this regard."
Lebanon's new national unity government has given Hizbullah and their allies veto power over all major decisions and also upheld Hizbullah's right to retain its weapons.
The cabinet also formally approved diplomatic ties with Syria and the opening of a Lebanese embassy in Damascus.
Information Minister Tarek Mitri said that Lebanon's foreign minister has been entrusted with following up on the mechanism to set up the embassy. He did not set a time frame.
The move was yet another step in ending the long chill between the two estranged neighbors, who earlier this month agreed to establish full diplomatic ties for the first time since they gained their independence from France in the 1940s.
The agreement on diplomatic ties came during a landmark visit last week by Lebanese President Michel Suleiman to Damascus for talks with Syrian President Bashar Assad. It was the first visit by a Lebanese head of state in three years.
During the visit, the two countries also agreed to negotiate the demarcation of their border, a standing Lebanese demand from its longtime dominant larger neighbor. Syria controlled Lebanon for nearly 30 years until its direct hold was broken in 2005.