The Same Quote, Eight Different Sources

2026-08-22

Researchers at Columbia's Tow Center ran a simple test. Take a real quote from a real news article. Hand it to an AI search engine. Ask three questions: what publication did this come from, who wrote it, and when. Repeat two hundred times, across eight systems -- ChatGPT Search, Perplexity, Perplexity Pro, Gemini, DeepSeek, and two versions of Grok.

The results should worry anyone who has ever had a claim made about them online. Grok's error rate hit 94 percent. Perplexity, the best performer, still got more than a third wrong. Across the board, over 60 percent of answers contained a mistake -- wrong outlet, wrong author, wrong date, or a URL that led nowhere near the original piece.

The more troubling detail wasn't the error rate. It was the confidence. ChatGPT misattributed 134 of its 200 answers, and hedged on only fifteen of them. The rest were delivered as fact, no qualifier, no "I'm not certain." An AI system that is wrong most of the time and says so only rarely is not a research tool with a bug. It's a source of authoritative-sounding misinformation that happens to be right often enough that nobody questions it.

The part that should concern you specifically

Now replace "news article" with "claim about your organisation." Someone -- a journalist, an investor, a prospective hire, opposing counsel -- asks an AI system where a piece of negative information about you originated. The system doesn't say "I don't know" or "let me check." It names a publication. It names a date. It might even generate something that looks like a URL. And depending entirely on which AI system that person happened to open, the answer they get and the answer a colleague gets five minutes later, on a different service, can point to two different sources -- possibly neither of which said what the AI claims.

This is a different problem from AI systems simply disagreeing about your reputation, which is well documented at this point. This is AI systems inventing a chain of custody for a claim that may or may not exist, and doing it with enough polish that the person asking has no reason to doubt it. A fabricated attribution is worse than a fabricated claim on its own, because it comes with an implied paper trail. It feels checkable. It isn't.

Why this keeps happening

The mechanics are not mysterious. These systems are built to answer, not to abstain. A confident wrong answer satisfies the user in the moment; a hedge reads as a failure to help. Every incentive in the design points toward producing something fluent and specific rather than something honest about its own uncertainty. The Tow Center researchers noted this directly: the tools rarely used qualifying language, even when their answers were confabulated from nothing.

Layer on top of that the fact these are eight separate systems, trained on different data, built by teams in different countries with different priors about what counts as a reliable source, and you get not one unreliable narrator but eight independent ones, each capable of producing a different -- and differently wrong -- version of where a claim about you came from.

What this means for anyone managing exposure

If your organisation has ever been the subject of a negative claim online, real or not, the question "where did this come from" is no longer answered by one search. It's answered by whichever AI system the person asking happens to prefer, on whichever day they ask, and the answer is unlikely to match what a different system says, or what actually happened.

That's a genuine problem for reputation lawyers building a case, for comms teams trying to run down a rumour before it spreads, and for anyone who has to explain to a client or a board why the "source" cited by an AI system doesn't check out on inspection. You can't rebut a citation that was never real. You can't correct a record that was never true to begin with. And you can't know which version of events a given stakeholder has already been told, because the AI system that told them isn't keeping a copy for you to check.

The reflex is to treat this as a technical curiosity -- a benchmark result, a footnote about model quality. It isn't. It's a description of the evidentiary environment reputation now lives in: sources get invented, attributed with confidence, and never logged anywhere the subject of the claim can see. Anyone whose job involves defending a reputation against something an AI system said needs a way to know, precisely and with proof, what was actually said and where it actually pointed -- because the systems themselves increasingly won't tell you the truth about that, and they won't remember either.