While Washington, Brussels and Beijing are trying to formulate regulations that can keep pace with reality, AI labs in Silicon Valley continue to race ahead.
The past 24 hours have offered a troubling snapshot of the field and, above all, the industry behind it: a senior researcher resigning with a blunt warning of an existential collapse, a morally questionable mathematical breakthrough involving thousands of autonomous agents and a social media giant releasing an agent that can purchase products for you directly with access to your bank account.
Gallery
Meta CEO Mark Zuckerberg, Anthropic CEO Dario Amodei, OpenAI CEO Sam Altman
(Photo: Getty Images, AP, AFP)
The first shock comes from the company that has defined itself as the safest and most responsible lab in the field. Jacob Coxon, a 27-year-old researcher who spent the past three years working at both OpenAI and Anthropic, announced his resignation from the latter while issuing a stark warning. According to Coxon, the two companies are not acting responsibly and are racing toward superhuman AI capable of self-improvement, while "gambling with our lives.”
Coxon claims that industry leaders privately express deep concern that the systems could lead to humanity’s destruction before the end of the decade. He puts the odds at no less than around 10%, but says they publicly reassure investors.
At Anthropic, he claims, executives are well aware of the potential future consequences but are trapped in a familiar paradox: the fear that another, less cautious company will get there first. The response from the company’s senior safety officials, who partially acknowledged the concerns while defending the continuation of the research, highlights Silicon Valley’s structural irony, in which every brake mechanism is treated as a competitive obstacle.
Shortly afterward, OpenAI demonstrated why researchers are so concerned. The company reported Tuesday night that an internal model had managed to crack a key aspect of the three-dimensional Navier-Stokes equations, one of the seven Millennium Prize Problems in mathematics, each carrying a $1 million prize.
The equations, formulated in the 19th century and describing the movement of liquids and gases, have occupied scientists for decades over the question of whether smooth solutions can break down within a finite amount of time. They are highly important equations that could help with calculations in aeronautics and meteorology and in estimating potential damage from floods or tsunamis, for example.
AI apps
(Photo: Getty Images)
To reach the result, OpenAI did not rely on a single human flash of insight. Instead, it deployed about 10,000 AI agents simultaneously for roughly 88 hours, at a computational cost of millions of dollars. The system produced a 165-page proof that was verified in the Lean formal programming language.
But behind the scientific achievement, an ugly credit dispute immediately erupted involving researchers from New York University and Anthropic, who claimed that the idea was based on their parallel research. The question of whether user data was used for training also remained without a clear answer.
While competing projects such as Google DeepMind’s AlphaProof have focused on solving mathematical problems from competitions under close academic supervision, here brute-force computing power on an unprecedented scale was deployed to establish facts on the ground before the scientific community had time to blink.
Meta isn’t giving up either
At the other end of the spectrum, in the consumer space, Mark Zuckerberg’s Meta announced the launch of “Muse,” a personal AI agent developed under the leadership of Alexandr Wang. Muse is far removed from a standard chatbot: It operates in a dedicated virtual machine in Meta’s cloud, browses the web using an independent browser, runs in the background even when the user is offline and remembers personal preferences over time.
Beyond conversations, Muse is equipped with a digital wallet based on Stripe Link, which generates one-time credit cards and enables the agent to make purchases independently. To ease concerns over malicious actions or critical errors, Meta has paired Muse with an isolated control system called “Sentinel,” whose role is to approve or block access to the web and sensitive operations.
Compared with similar personal agents, such as OpenAI’s Operator, Google’s Astra and agent systems now being developed in China by Baidu and Alibaba, Meta is betting on a deep combination of episodic memory, autonomous purchasing capabilities and eventual integration with its smart glasses.
The connection between these three developments exposes the paradox facing the entire industry. On the one hand, existential warnings from researchers inside the industry are becoming more concrete as models solve complex physical problems using armies of autonomous agents.
On the other hand, the labs are not slowing down. They are instead bringing those same agents directly onto users’ phones, along with access to their bank accounts. The conclusion is clear: The industry cannot be trusted to oversee these developments on its own. What is needed is a coordinated global effort involving governments, researchers and industry, operating in concert and taking measured steps.