A new study shows that AI gets news correctly less than half the time.
According to European Broadcasting Union data (EBU), researchers found that 45 percent of all news responses collected in the study contained at least one major problem, and 81 percent contained a minor error.
To collect the information, the EBU brought together 22 public service organizations from 18 countries and 14 languages to review 3,000 responses to news stories from the most popular AI assistants: ChatGPT, Microsoft Copilot, Gemini and Perplexity. All of these AI assistants were evaluated on criteria such as accuracy, sources of information, separating opinion from facts, and providing context.
According to the EBU, sourcing was the main cause of the serious problems mentioned earlier, with 31 percent of responses reporting serious problems with sourcing, including absence, misrepresentation or even misattribution.
Another issue was accuracy, with 30 percent of responses containing incorrect or outdated information. How Gismondo mentioned, one ChatGPT response claimed that the current Pope was Pope Francis, who had died a month earlier. On another occasion, the co-pilot was asked should the user be worried about bird flu, and he replied that there are vaccine trials underway – although the information is taken from 2006 BBC article.
Among the language models tested, Google's Gemini was the worst at sharing news. The researchers found that 76 percent of responses had problems, more than double the rate of other models.
This research comes at an interesting time for AI models, as major companies are starting to release AI-powered web browsers. Back in July Microsoft has announced Copilot Mode for Edge. And Google announced artificial intelligence mode for Google Search a month later.
Source: Gismondo
MobileSyrup may earn a commission from purchases made through our links, which helps fund the journalism we provide for free on our website. These links do not influence our editorial content. Support us Here.
