How Much Can We Trust AI?

Ask an AI chatbot about a war, an election or a scientific controversy and it will give you a neat answer in seconds. The sentences are clear. The tone is calm. It sounds as though someone has weighed the evidence and reached a considered conclusion.

But what if the chatbot has simply inherited somebody else’s version of events?

The edit war you never see

In January, I wrote about the risk of Wikipedia’s editorial biases finding their way into AI. A recent piece from HonestReporting brings that question closer to home. What happens when a dispute over the wording of a Wikipedia article becomes part of the answer a chatbot gives you today? 

HonestReporting points to a Toronto event advertised as a “Free Palestine Wiki Edit-a-Thon.” Organisers offered to train participants to edit Wikipedia and then work together on articles. There is nothing inherently wrong with teaching people to edit an encyclopedia. But when an event is organised around an explicit political cause, it’s fair to ask how the resulting edits will be checked for balance. 

Wikipedia has already had to referee disputes over articles on Israel and the Palestinians. In January 2025, its Arbitration Committee barred eight editors from working on the topic. Reporting at the time identified six as pro-Palestinian and two as pro-Israel. The bans show that Wikipedia has a way to deal with disruptive editing. They also show how fiercely people fight over what goes into an encyclopedia many readers take at face value. 

How the bias travels

There are two ways a source can influence an AI answer. Its text can be part of the material the model learned from during training. Or an AI tool with web access can find the page when you ask your question and use it to compose a response. HonestReporting’s piece highlights that second route. It gives us reason to scrutinise the answers, although tracing any one answer to a particular edit takes more evidence. 

An outright falsehood is often the easiest problem to spot. Framing is harder. Which event gets the first paragraph? Whose account is presented as fact and whose is introduced as a claim? What happened the day before, and did it make it into the summary? Every sentence can be defensible on its own while the overall account leaves a misleading impression.

On Wikipedia, a curious reader can look at the footnotes, the editing history and the arguments on the discussion page. A chatbot can compress all of that into three tidy paragraphs. Unless you follow the links, you may never know there was an argument in the first place.

So how much should we trust it?

I use AI and will keep using it. It’s a quick way to get oriented on an unfamiliar subject. But on a disputed question, I want to see where the answer came from. If three citations all lead back to the same original claim, I’ve still got one source. And if the chatbot gives me a conclusion I’m already inclined to believe, that’s an especially good time to open the links.

The Israel example should concern us whatever our politics. The same mechanism can carry someone’s preferred account of an election, a public health controversy or a government’s record. The more people rely on a single answer, the more it matters who shaped the sources behind it.

The next time AI gives you a confident answer on a question people are still fighting over, ask it one more question. What evidence would make you change that answer? Then see whether the evidence it gives you is worth trusting.

Further reading

• How Wikipedia’s Anti-Israel Edits Influence AI Answers— HonestReporting
• Wikipedia, Bias, and the Poisoning of Our Future Knowledge⁠ — Grappy’s Soapbox