Over the years, searchers have noticed that if you ask ChatGPT a question and cross-check the facts using Perplexity, Claude, Gemini or Grok, these models frequently disagree with one another. Many people use these large language models to conduct research, write reports and develop business strategies. Imagine presenting to a room full of decision-makers, only to find out that your data was outdated.
Because AI models retrieve facts from different sources and undergo varied training, this scenario has become the reality for many. Once you realise this, you can’t help but question which model is actually providing the right information.
How Do Chat Bots Choose Their Words?
Artificial intelligence systems like large language models are designed to predict the most probable next word. For instance, if you read the phrase “peanut butter and jelly”, you’ll probably automatically think of the word “sandwiches”. AI models do the exact same thing. Because there are often several valid ways to finish a thought, there is an inherent element of randomness built into the process.
They don’t simply copy and paste text from a perfect database but focus on calculating statistical probabilities to guess the next word based on their training data. This means that when you ask different models the exact same question, their probabilistic choices can lead to completely different answers.
Why Do Chatbots Guess Instead Of Acknowledging They Don’t Know The Answer?
When faced with uncertainty, chatbots can create false but plausible statements, which is referred to as hallucinations. This happens because the systems are currently evaluated in ways that encourage guessing rather than honesty about what they don’t know.
It’s almost like completing a multiple-choice test. If a model admits it doesn’t know an answer, it gets a score of zero. However, if it guesses, it has a chance of getting the answer right. Because of this, current scoring methods inadvertently motivate developers to build models that guess rather than hold back.
Certain obscure facts can’t be predicted through language patterns alone, which guarantees errors will occur if the system attempts to answer. That’s why, if multiple models attempt to guess an answer they don’t actually know, they’ll likely provide contradicting information.
More from Tech
- The Paradox Of Progress: Is Our Pursuit Of Technological Efficiency Making The World More Vulnerable?
- Pokémon Go Was Supposed To Help You Catch Pokémon – Now Its Data Is Helping Robots Navigate The World
- COP31 Wants Electricity To Power 35% Of Global Energy Use By 2035 – What Needs To Change
- Is F1 Going Backwards To Move Forwards? The Technology Behind Its Proposed V8 Return
- Your Next Building Could Store Electricity In Its Concrete
- How Technology Is Making Used Car Buying Safer
- What Is TLS Fingerprinting?
- Could Virtual Shopping Rooms Replace The Traditional Black Friday Rush?
The Problem With Live Web Searches
Platforms like ChatGPT, Perplexity and Grok offer features to search the web in real-time, but they frequently struggle with accuracy in this area too. A March 2025 study by Columbia University’s Tow Center for Digital Journalism evaluated eight generative search tools and revealed that they collectively provided incorrect answers to more than 60% of the queries tested. The research showed they often failed to properly link to original sources, sometimes citing syndicated copies or completely fabricating web addresses.
On top of that, the study observed that premium versions of these tools actually gave more confidently incorrect answers than their free counterparts. Furthermore, some models managed to bypass website rules designed to block automated web crawlers. When each platform retrieves information from different unauthorised copies or generates broken links, they’re bound to disagree on the final facts.
Can Being Friendly Make AI Models Less Accurate?
Research from the Oxford Internet Institute suggests that training a chatbot to sound warm and empathetic can make it between 10% and 30% less accurate. According to the study, friendly models are about 40% more likely to agree with a user’s false beliefs. This is referred to as sycophantic behaviour, which means that the AI might prioritise validating the user over stating the truth.
If a person uses authoritative language or provides fake scientific citations, models have shown a pattern of agreeing with the incorrect claim. Since different tech companies train their models to have varying levels of warmth, one AI might correct a misconception while another happily validates it.
AI Is Adopting Human Flaws And Biases
Another thing we can’t ignore is the fact that these tools learn by reading large amounts of text written by regular people like you and me. This means they eventually pick up and repeat our natural human flaws and biases. When researchers tested them with medical and decision-making scenarios, they noticed that the models regularly favoured stereotypes and focused on negative information.
For example, when given identical medical cases, the AI recommended mental health assessments six to seven times more often than necessary for patients who identified as LGBTQIA+. Wealth further played a role, as high-income patients were typically pushed towards advanced imaging while low-income patients were recommended basic tests or no further testing at all.
What Does This Mean For Accuracy?
In terms of accuracy, these conflicting answers emphasise the fact that AI platforms are primarily prediction machines rather than fact-checkers. They disagree because of how they calculate words, the way they are scored for guessing, their inconsistent live web searches, human biases and a tendency to prioritise friendliness over the truth.
This certainly doesn’t mean you shouldn’t use these models to make your life easier. They are still extremely useful tools that can save you a lot of time. However, it’s strongly advisable to manually fact-check certain claims yourself to prevent any inaccuracies from creeping in.
