AFP Probe Reveals AI Chatbots Are Unreliable During Breaking News
AFP
Islamabad: As tensions flared between India and Pakistan in recent weeks, a surge of misinformation swept across social media platforms. Many users, in search of quick verification, turned to AI-powered chatbots — only to be misled by inaccurate responses, exposing the growing risks of relying on artificial intelligence for real-time fact-checking.
An investigation by AFP has found that several leading AI tools, including xAI’s Grok, OpenAI’s ChatGPT, and Google’s Gemini, frequently produced false or misleading information, particularly in fast-moving news cycles where verified facts were still emerging.
During the recent conflict, Grok misidentified old footage from Sudan’s Khartoum airport as evidence of a missile strike on Pakistan’s Nur Khan Airbase. In another instance, a video of a burning building in Nepal was wrongly attributed to Pakistan’s military response to Indian airstrikes. These high-profile errors highlight the limitations of AI tools in processing and verifying complex, evolving news events.
“The growing dependence on Grok as a fact-checking tool comes amid significant cutbacks in human moderation by major platforms,” said McKenzie Sadeghi, a researcher with disinformation watchdog NewsGuard. “Our ongoing analysis shows that AI chatbots consistently fall short in delivering accurate news, especially during unfolding crises.”
A broader study conducted by NewsGuard revealed that ten of the most popular AI chatbots routinely repeated disinformation narratives, including propaganda linked to Russia and misleading statements about Australia’s elections. The Tow Center for Digital Journalism at Columbia University also noted that these tools rarely refuse to answer unverifiable questions, often defaulting to speculation instead.
In one alarming case, AFP fact-checkers found that Google’s Gemini confirmed the authenticity of an AI-generated image of a woman — and even invented detailed background information about her identity and location. Grok, on the other hand, falsely verified a viral video purporting to show a giant anaconda in the Amazon River, citing fictitious scientific expeditions as proof.
The growing use of AI for fact-checking coincides with Meta’s controversial decision to phase out its third-party fact-checking programme in the U.S. The company has shifted toward a “Community Notes” model — inspired by X (formerly Twitter) — that relies on user contributions to flag misinformation. However, experts have expressed skepticism about the effectiveness of such community-driven systems in combating complex and coordinated falsehoods.
Human-led fact-checking remains a contentious issue in the U.S., where some conservative groups have accused fact-checkers of bias and censorship. These allegations have been strongly denied by professionals in the field. AFP continues to partner with Facebook on a global fact-checking initiative, operating in 26 languages across regions including Asia, Latin America, and Europe.
Meanwhile, questions have also been raised about potential political influence over AI-generated content. Grok recently came under scrutiny after it generated posts referencing the far-right “white genocide” conspiracy theory in unrelated queries. xAI attributed the incident to an “unauthorised modification” of the system’s instructions — a claim met with widespread skepticism.
AI researcher David Caswell directly queried Grok about the modification, receiving a startling response that named Elon Musk — founder of xAI and owner of X — as the “most likely” source. Musk, a South African-born entrepreneur and vocal supporter of former U.S. President Donald Trump, has previously echoed controversial views on race and demographics, including the “white genocide” theory in the context of South Africa.
Angie Holan, director of the International Fact-Checking Network, warned of the dangers posed by biased or fabricated AI responses. “I’m deeply concerned by the way Grok and other chatbots are programmed to offer pre-approved answers, particularly on sensitive topics. The potential for manipulation is enormous,” she said.
As reliance on AI tools continues to rise, experts are urging caution and renewed investment in human-led verification systems to safeguard the integrity of public discourse — especially during moments of geopolitical crisis.
