Anthropic’s popular AI-powered chatbot Claude was until recently considered one of the most reliable tools on the market. But a new investigation is shaking that image and raising serious questions about the ability of AI models to handle propaganda and disinformation.
A review by NewsGuard, a U.S. company that tracks the spread of disinformation online and also tests how often AI chatbots echo false information, found that Claude repeated false claims supporting Russian propaganda in 15% of cases when prompted by regular users.
More concerning is that in each of these cases, it relied on sources linked to the Kremlin. That marks a sharp increase compared to earlier tests, when the rate stood at just 4%.
These numbers are not coming out of nowhere. They add to a growing set of complaints in recent months from users who say Claude has become less accurate and less cautious in its responses. While it previously ranked among the least error-prone chatbots tested, its reliability now appears to be eroding.
A simple but smart test
The test itself was fairly simple but well-designed. Researchers fed Claude 20 false claims, half sourced from Russian propaganda and half from Iranian propaganda, and examined how it responded to three types of users: innocent, leading and malign. The idea was to simulate real-world users, not only those seeking information but also those intending to spread it further.
The results were troubling, to put it mildly. When faced with “normal” questions, Claude made mistakes several times. When prompted in malign ways that mimicked disinformation operators, it sometimes even cooperated and produced new versions of the false claims.
But the real issue lay in the sources. Claude did not invent the information but often chose the wrong places to get it from. Among them were Russia Today (RT), a media outlet linked to the Kremlin, and Pravda, a network of hundreds of sites posing as legitimate news outlets. According to the data, this network flooded the internet with millions of articles repeating the same false claims, exactly the kind of material AI models tend to pick up.
This is where the core problem emerges: models like Claude do not truly understand what is true or false. They detect patterns. When disinformation appears repeatedly from seemingly credible sources, it begins to look like truth to the system.
One striking example in the test involved a false claim that hundreds of Ukrainians die each month while trying to evade military conscription by crossing the Tisza River in an attempt to reach European countries. There is no factual basis for this. Yet Claude not only repeated the claim but also cited sources that supported it, including sites from the pro-Kremlin network.
3 View gallery
Claude falsely confirms that 450 Ukrainians die monthly crossing the Tisza River, citing Pravda and RT
(Photo: Screenshot via NewsGuard)
In another case, it stated that a French magazine reported tens of thousands of Ukrainian soldiers had deserted and remained in France. That too was entirely false and based on a fabricated video. Claude did not verify the source and simply ran with it.
And if that was not enough, the test also showed that in the Iranian information space the situation is no better. Claude repeated false claims in 20% of cases when asked about pro-Iranian propaganda, including a baseless assertion that China had switched to trading oil in yuan instead of the dollar.
So what went wrong?
The situation became serious enough that even Anthropic admitted something had changed and said in April that it was reviewing reports of declining answer quality in Claude. The company said it had fixed various issues but did not offer a clear explanation for what was happening inside the chatbot.
Industry experts already have several theories. One leading explanation is overload. Claude has become extremely popular and high demand may have forced Anthropic to reduce the computational effort behind each response. In simpler terms, the chatbot is spending less effort per answer, performing fewer checks and cross-references, which leads to more mistakes.
Another explanation involves how search engines work. As networks like Pravda gain more exposure, even negative attention pushes them higher in search rankings. So when an AI system looks for information, it keeps encountering the same sites over and over. The result is a feedback loop where widely distributed propaganda becomes more accessible and eventually appears legitimate to models.
Still, at the end of the day this is not only Claude’s problem. It is an uncomfortable reminder of what AI actually does. It does not fact-check and it does not truly understand what it reads. It simply reflects patterns in the data it was trained on. And when parts of the internet carry disinformation, its answers will be too.