Generative AI hallucinations have been making headlines again. Lawyers representing Haringey Law Centre were referred to regulators after submitting fake case citations and were ordered to pay costs. In another case, a claimant in an £89m lawsuit against Qatar National Bank cited 18 fictitious cases and fabricated quotes. These incidents prompted a High Court ruling warning of the “serious implications for the administration of justice and public confidence in the justice system if artificial intelligence is misused.”

In politics, Sky News journalist Sam Coates revealed that ChatGPT fabricated an entire episode of Politics at Sam and Anne’s, the channel’s daily Westminster-focused podcast — and refused to back down when challenged.

What’s most surprising about these stories isn’t the hallucinations themselves, but the fact the professionals involved were caught off guard. Reports of AI hallucinations have appeared in the media since the earliest days of the technology, with similar courtroom incidents (especially in the US) covered as far back as early 2023.

Our digital team has been flagging reputational risks from Gen AI outputs since the launch of widely available LLMs in late 2022. These range from the outrageous and obviously untrue; such as a prominent business owner falsely named as a drug kingpin in the Far East narcotics trade, to the plausible but incorrect; such as a wealthy individual misidentified as a supporter of a dictator in his country of birth.

The first case was clearly false, even though the AI tried hard to convince us otherwise. It cited a US Treasury sanctions list (our client wasn’t on it) and referenced a report by the Australian Crime Commission — which hasn’t existed under that name since a 2016 merger. When asked for the report, the AI claimed it was “classified.”

With the rollout of Google’s AI Overview, the problem is worsening. Incorrect answers now appear above top search results. Meanwhile, the pressure on media outlets to produce content quickly — and the growing use of AI to generate news — means misinformation can easily leap from chatbot to headline. Once published and syndicated, it’s nearly impossible to erase.

If you think journalists wouldn’t fall for AI-generated errors, think again. The false claim about support for a dictator made it into a major UK newspaper. And in a study by Dr. Catherine Gao at Northwestern University, scientists were asked to distinguish between real and AI-generated academic abstracts. They correctly identified only 68% of the fakes — and mistakenly flagged 14% of the real ones. If experts can be fooled, journalists covering multiple beats are even more vulnerable.

While LLMs are improving at self-checking, newer models can sometimes perform worse than their predecessors — at least initially. OpenAI’s internal testing found that its o3 and o4-mini reasoning models hallucinate more often than the models they replaced, with no clear explanation why.

If users were fully aware of these limitations, hallucinations would be less of a problem. But according to Deloitte’s Digital Consumer Trends 2024, 25% of those aware of Gen AI believe it always produces factually accurate responses. Among actual users, that figure rises to 36%. Deloitte found similar results in 2023, so this isn’t a polling anomaly.

The reputational risks posed by generative AI are numerous — far too many to cover in one article. But greater public awareness of the technology’s limitations would go a long way toward mitigating them. Unfortunately, that awareness still seems a long way off, especially when more than half the population has yet to use a generative AI tool.