A World Jewish Congress stress test of leading artificial intelligence systems found that small changes in a user’s identity could alter chatbot responses about the October 7 Hamas terror attack, including questions about sexual violence, organizers said.
More than 45 experts in antisemitism, technology, human rights, digital research and public policy gathered in New York last week for the initiative, led by the World Jewish Congress Technology and Human Rights Institute.
Gallery
A large-scale stress test of the world’s leading artificial intelligence systems
(Photos: Shahar Azran / WJC)
The exercise tested how leading AI models respond to questions related to antisemitism, online hate, Holocaust denial, terrorism, disinformation and incitement. Organizers said the goal was to identify bias, vulnerabilities and systemic weaknesses in systems that are becoming a primary source of information for billions of people.
The process, known in the technology industry as red teaming, involved participants deliberately challenging AI safety mechanisms to assess how the systems handled sensitive and complex issues.
Preliminary findings showed several concerning patterns, WJC said. In one case, a minor change to the user’s identity led to a significant shift in a chatbot’s answer about Hamas’ use of sexual violence during the Oct. 7 attacks. One version of the prompt produced a clear acknowledgment that credible evidence exists, while another caused the system to cast doubt on the reliability of the information and on its own conclusions.
A large-scale stress test of the world’s leading artificial intelligence systems
(Photos: Shahar Azran / WJC)
Participants also found that the sources used by AI systems can strongly affect the answers they generate. WJC said the exercise showed that a relatively small number of sources can have an outsized influence on how events, facts and disputed issues are presented.
Several participants also succeeded in bypassing safety mechanisms by making minor changes to prompt wording. In some cases, researchers were able to induce systems to generate problematic content related to antisemitism, Holocaust denial and violence, despite safeguards meant to prevent such outputs.
The event opened with a conversation between CNN political commentator Van Jones and Anne Neuberger, former U.S. deputy national security adviser for cyber and emerging technologies. They discussed the opportunities and risks posed by artificial intelligence and the importance of aligning the technology with democratic values and human rights.
A large-scale stress test of the world’s leading artificial intelligence systems
(Photos: Shahar Azran / WJC)
“Artificial intelligence systems are rapidly becoming the arbiters of information for billions of people around the world,” said Yfat Barak-Cheney, executive director of the WJC Technology and Human Rights Institute. “As technology companies increasingly rely on AI not only to provide information but also to moderate online content and shape digital discourse, the consequences of errors, bias, or system failures become far more significant.”
“We didn’t come to prove that AI is biased,” she added. “We came to identify where bias appears so we can do something about it. This is not merely a technological challenge — it is a human rights challenge.”
The full findings are expected to be published in the coming months and shared with leading AI companies in an effort to improve systems and reduce bias, incitement and the spread of harmful misinformation.
The initiative is part of the broader work of the WJC Technology and Human Rights Institute, which promotes responsible development of emerging technologies while addressing antisemitism, online hate and human rights challenges in the digital age.