AI chatbots battle with factual inaccuracies and distortions when summarizing information tales, analysis from the BBC has discovered. The research, which examined whether or not OpenAI’s ChatGPT, Google Gemini, Microsoft Copilot, and Perplexity can precisely summarize information, discovered greater than half of all of the AI-generated output had “vital problems with some type.”
As a part of the research, the BBC requested ChatGPT, Copilot, Gemini, and Perplexity to supply summaries of 100 BBC information articles, whereas journalists reviewed their solutions. Along with discovering main points in 51 p.c of responses, the BBC discovered that 19 p.c of solutions citing the BBC included incorrect statements, numbers, and dates. In the meantime, 13 p.c of quotes from the BBC have been “both altered from the unique supply or not current within the article cited.”
The research highlighted some examples, together with Gemini incorrectly stating that the UK’s Nationwide Well being Service (NHS) “advises individuals to not begin vaping, and recommends that people who smoke who wish to give up ought to use different strategies.” Nevertheless, the NHS really recommends vaping to give up smoking. One other instance: in December 2024, ChatGPT claimed Ismail Haniyeh was a part of Hamas management though he was assassinated in July 2024.
Total, the research discovered Gemini’s responses “raised essentially the most considerations,” as 46 p.c have been “flagged as having vital points with accuracy.” The Verge reached out to OpenAI, Google, Microsoft, and Perplexity with requests for remark however didn’t instantly hear again.
In a response to the research, Deborah Turness, the CEO of BBC Information and Present Affairs, referred to as on tech firms to handle points with inaccuracy. “We dwell in troubled occasions, and the way lengthy will or not it’s earlier than an AI-distorted headline causes vital actual world hurt?” Turness wrote. “We’d like different tech firms to listen to our considerations, simply as Apple did. It’s time for us to work collectively — the information business, tech firms — and naturally authorities too has a giant function to play right here.”
