As artificial intelligence industry leaders warn of potentially existential dangers, developments on the battlefield are moving in another direction: Washington and Moscow are working together in an unexpected effort to weaken a series of proposed UN restrictions on lethal autonomous weapons, including provisions intended to keep humans involved in life-and-death decisions.
According to a new Washington Post report, the United States and Russia spent roughly 15 hours in closed-door negotiations in Geneva, without observers present, pressing to remove key requirements from a draft agreement.
Russian President Vladimir Putin and US President Donald Trump
(Photo: AP Photo/Alexander Zemlianichenko, Pool; Andrew Harnik/Getty Images)
The two powers each brought teams of about 10 lawyers to the negotiations, the report said.
Among the provisions removed were requirements that autonomous weapons operate predictably and reliably, that ethical considerations be taken into account in their use and that a human approve military targets identified by algorithms before a strike is carried out.
The move puts Washington and Moscow at odds with the more restrictive approach traditionally promoted by European countries and illustrates how the pursuit of technological and military advantage is colliding with attempts to prevent warfare from becoming fully automated.
Human rights groups described the weakening of the draft as “death by a thousand cuts,” warning that stripping out oversight provisions could pave the way for warfare in which machines operate without real-time human involvement, increasing risks to civilians and making legal responsibility more difficult to establish.
Drawing the red line
Leading militaries have increasingly incorporated advanced algorithms into intelligence analysis and target generation.
The Pentagon used advanced models to identify nearly 1,000 potential strike targets during operations against Iran, while the IDF uses AI-based systems such as “The Gospel” to process intelligence and generate large numbers of potential targets.
In both cases, the systems have been described as decision-support tools rather than systems independently authorized to open fire.
But the line between a machine recommending a target and a machine effectively determining that target is becoming increasingly difficult to define, particularly as battlefield response times shrink to seconds.
China is developing autonomous drone swarms and armed robotic systems as part of a broader approach to algorithm-driven warfare, while the United States is advancing its Replicator initiative, aimed at deploying thousands of relatively inexpensive autonomous systems as a response to China’s numerical military advantage.
At the same time, the Trump administration has made clear that it opposes international oversight that it believes could restrict technological innovation and has instructed the Pentagon to revise its guidelines for military use of artificial intelligence.
That approach helps explain Washington’s opposition to a binding UN agreement imposing stricter restrictions.
Operational advantage over ethical principles
Past international arms-control agreements, from the Chemical Weapons Convention to bans on anti-personnel mines and cluster munitions, were built around attempts to establish a moral red line.
Autonomous weapons pose a different challenge because they are not a single category of weapon. AI software can be incorporated into existing platforms such as drones, cruise missiles and automated gun systems.
As major powers resist surrendering potential military advantages in favor of stricter ethical restrictions, the Geneva effort could ultimately result in little more than broad, nonbinding declarations while responsibility for pulling the trigger shifts gradually from a human operator to lines of code.
That raises a fundamental accountability problem.
Unlike a soldier, whose conduct can be examined through a command structure and established orders, artificial intelligence still suffers from two major limitations.
The first is the so-called black box problem. In many cases, it can be difficult to determine exactly why an AI model produced a particular output or reached a specific conclusion.
That creates obvious difficulties when investigators need to reconstruct the chain of reasoning behind a lethal decision.
The second, potentially more serious problem, is reliability and the phenomenon known as hallucination.
Even modern AI models can generate information that does not exist. In ordinary civilian applications that may produce an incorrect answer. On the battlefield, where a model may be expected to distinguish an enemy target with absolute precision, the consequences can be far more serious.
An autonomous system that incorrectly identifies civilians as a military target could make an irreversible decision before a human has an opportunity to intervene.
It remains unclear whether the changes pushed by the United States and Russia will ultimately be adopted or whether they could derail the UN initiative altogether.
But without a legal framework defining when and how AI systems may move beyond assisting human decision-makers to making lethal decisions themselves, the boundary between human judgment and automated warfare will continue to erode.