ChatGPT Hallucination Statistics: Every Published Number

What This Page Answers

People land on a page like this wanting one of a few things: the current headline hallucination rate for ChatGPT and its rivals, a comparison of hallucination rates across GPT, Claude, Gemini, and Grok, evidence for a specific domain like law or medicine, or citable numbers for a research paper, policy brief, or news article. This page is built to answer all of those, with every statistic traced back to its original source.

One warning up front: hallucination rate is not a fixed property of a model the way a parameter count is. The same model can score under 2% on one benchmark and over 45% on another, depending on whether it’s summarizing a document it can see or answering from memory, whether web search is enabled, and how strictly the benchmark defines a hallucination. Any single “X% hallucination rate” headline you see elsewhere is almost certainly describing one specific benchmark, not a universal truth about the model.

How This Page Was Built

Key Findings Across Every Study Reviewed

The clearest pattern across every domain studied is a split between grounded and ungrounded tasks. When a model is given a source document and asked to stay faithful to it, the best current systems hallucinate in the low single digits. When a model has to answer purely from memorized training data, with no document or web search to check against, hallucination rates jump into the double digits and frequently exceed 40%. That single distinction explains most of the apparent contradictions between different “hallucination rate” headlines you may have seen.

The second pattern is domain risk. Specialized, high-stakes domains, law, medicine, and academic citation, consistently show higher hallucination rates than general benchmarks. That is the opposite of what many users assume, since these are exactly the domains where a fabricated case citation or a fake journal reference does the most damage. For the education-policy side of this same problem, see our companion review of AI detection policies at 50 leading U.S. universities, and for how fabricated citations feed into misconduct cases, see our AI academic misconduct statistics from every published survey.

Charts: Hallucination Rates by Model and Task

Master Table: Every Benchmark Number in One Place

This table consolidates every model-level hallucination figure referenced on this page. “Self-reported” means the number comes from the model vendor’s own system card or blog post. “Independent” means it comes from a third-party academic benchmark or research team with no commercial stake in the result.

Rates for late-2025 and 2026 models are drawn from vendor system cards and third-party benchmark aggregators; because release cadence has been roughly monthly, treat any single number as a snapshot rather than a permanent ranking. See the Sources section for direct links to every underlying benchmark and paper.

SimpleQA, TruthfulQA, and the Accuracy-vs-Hallucination Tradeoff

SimpleQA is a benchmark of short, fact-seeking questions with a single verifiable answer, an event date, a birthplace, a publication year. It is designed to be hard to game with retrieval tricks and easy to score automatically. On this benchmark, without web search enabled, OpenAI’s own GPT-5 system-card data shows a 47% hallucination rate against 46% accuracy, meaning the model is roughly as likely to confidently invent an answer as to get one right. GPT-4.5 posted a 37.1% hallucination rate on the same test.

OpenAI’s own explanation of this pattern, published in its research on why language models hallucinate, is that older reasoning models like o4-mini can post a slightly higher raw accuracy score than newer models while producing a much higher hallucination rate, because the older model guesses more often instead of abstaining. A model that answers every question will look impressive on an accuracy leaderboard while quietly generating far more fabrications than a model that says “I don’t know” when it’s genuinely uncertain.

The gap between GPT-4’s 3% Vectara summarization rate and GPT-5’s 47% SimpleQA rate is not a contradiction; it is two different tasks. Summarization gives the model a document to stay faithful to. SimpleQA asks it to recall a fact with nothing to check against. That distinction should be the first thing any reader checks before quoting a single hallucination percentage.

Legal Citation Hallucinations

Legal research is the domain with the most rigorous, most cited hallucination research to date, largely because of a string of sanctioned court filings that began with the 2023 Mata v. Avianca case, where a submitted brief cited entirely fictitious judicial decisions generated by ChatGPT. For a full accounting of that case and every subsequent sanctioned filing, see our tracker of AI detection lawsuits and their outcomes.

Stanford RegLab and the Stanford Institute for Human-Centered AI ran the two most-cited empirical studies. The first, led by Dahl and colleagues, tested general-purpose LLMs including GPT-4 and Llama 2 across roughly 15,000 federal court queries and found hallucination rates ranging from 69% to 88%, with GPT-4 at the lower end near 58% in the underlying citation analysis and Llama 2 at the higher end near 88%. The researchers also found hallucination rates were systematically higher for lower federal courts than for the U.S. Supreme Court, and higher for less-prominent cases, meaning the tail of American case law is the least reliably represented.

The second Stanford study, led by Magesh and colleagues, tested commercial legal AI research tools, Lexis+ AI, Ask Practical Law AI, and Westlaw AI-Assisted Research, that use retrieval-augmented generation against real case databases specifically to avoid this problem. Even these purpose-built tools hallucinated in 17% to 33% of tested queries, well above the “hallucination-free” language some vendors had used in marketing at the time. Accuracy also varied widely between products, with one tool answering accurately 65% of the time and another only 18% of the time on the same query set.

Because these numbers directly inform institutional policy, they connect to two of our companion reports: how universities are spending on AI detection tools in response to reliability concerns, and how those concerns have led some institutions toward banning AI detectors outright rather than trusting automated verdicts built on the same underlying technology that hallucinates.

Medical and Healthcare Hallucinations

Medical hallucination research splits into two categories: fabricated citations in AI-written medical content, and fabricated clinical facts inside AI-generated answers to health questions. Both are well studied and both show consistently elevated rates compared with general-purpose benchmarks.

The mental health citation study is worth a closer look because it shows how topic familiarity, not model version, is often the strongest predictor of fabrication risk. For well-studied conditions like major depressive disorder, GPT-4o fabricated only 6% of citations, with 64% of the remaining “real” citations fully accurate. For less-studied conditions like binge eating disorder and body dysmorphic disorder, fabrication rates jumped to 28% and 29% respectively, and accuracy among the real citations fell as low as 29%. Newer model versions did not clearly resolve this gap, suggesting the underlying limitation is training-data density on a topic, not just raw model capability.

Academic Citation Fabrication

The most widely cited cross-study analysis, by Walters and Wilder, pooled six separate peer-reviewed audits of ChatGPT-generated bibliographic citations across medicine, geography, and radiology. Citation fabrication rates in the individual studies ranged from 17% to 94%, with a pooled rate of 51% fabricated across all 732 citations reviewed.

Broader testing on GPT-3.5 vs. GPT-4 by the same research group found the newer model fabricated 18% of citations against 55% for GPT-3.5, and produced substantive errors, wrong volume, page, or issue numbers, in 24% of its real citations against 43% for GPT-3.5. A related DOI-specific analysis found that hallucinated DOIs were far more common in the humanities (up to 89.4%) than the natural sciences (about 29.1% invalid), reinforcing the same topic-density pattern seen in the medical citation research above.

These figures sit directly behind the growing concern over AI-generated research papers, tracked in our 2026 statistics roundup, and they are a core driver of the broader concerns documented in our academic misconduct statistics from every published survey. A fabricated citation is not just a factual error, in an academic submission it can constitute misconduct in its own right, independent of any AI-detector verdict.

Journalism and News Accuracy

The largest public-interest study of AI hallucination in news came from the European Broadcasting Union and the BBC in October 2025. Twenty-two public service media organizations across 18 countries, working in 14 languages, had professional journalists evaluate more than 3,000 AI assistant responses from ChatGPT, Copilot, Gemini, and Perplexity against news-specific accuracy, sourcing, and fact-versus-opinion criteria.

A separate academic analysis of AI-generated content in American newspapers, comparing 100 articles flagged as AI-generated against 100 human-written articles, found AI-labeled articles were 8.2 times more likely to contain at least one hallucinated claim, with rates of 26% to 41% depending on how a “hallucination” was scored, compared with just 5% for human-written articles. Typical errors included fabricated quotes, incorrect statistics, and misdated events, exactly the category of error a reader has no easy way to catch without independently verifying the underlying facts.

Software Package Hallucinations

Hallucination isn’t limited to prose. A 2025 USENIX Security study generated 576,000 code samples across 16 popular code-generating LLMs and two languages, Python and JavaScript, then checked every recommended software package against real package repositories. The average hallucination rate was 5.2% for commercial models and 21.7% for open-source models, for an overall average around 19.7%, and the researchers catalogued 205,474 unique hallucinated package names across the dataset.

This matters beyond code quality: researchers flagged “package confusion” as a genuine supply-chain attack vector, since a hallucinated package name is a prediction attackers can pre-register with real, malicious code before a developer downloads it. GPT-family commercial models were found roughly four times less likely to hallucinate packages than open-source alternatives, but the researchers noted that even a hallucination rate of a fraction of a percent, at the scale of tens of millions of daily code completions, still produces thousands of fictitious packages every day.

Why ChatGPT Hallucinates

OpenAI published its own explanation of the root cause in 2025, and the argument is more about incentives than about a technical bug. Most benchmarks are scored the way a multiple-choice exam is scored: a correct answer earns points, a wrong answer earns nothing, and a blank answer also earns nothing. Under that scoring system, a model that always guesses will out-score a model that says “I don’t know” whenever it should, because guessing has a positive expected value even when it’s frequently wrong.

A second driver is task type, covered throughout this page: grounded tasks with a document or search results to check against give the model something to be faithful to, while open recall tasks depend entirely on statistically likely-sounding completions drawn from training data, with no mechanism to verify the output before it’s generated. A third driver, seen clearly in the medical and academic citation research, is topic density: well-represented topics in training data produce far fewer fabrications than niche or specialized ones, even from the same model on the same day.

What Actually Reduces Hallucination Rates

Across every mitigation study reviewed for this page, one architectural choice consistently outperforms every prompting trick: grounding the model’s output in a real, retrievable source at generation time.

These compute-hungry mitigations are not free: extended reasoning and retrieval pipelines both increase the energy and hardware demand behind every query, a tradeoff explored in our broader look at AI data centers and the environment. Institutions weighing whether to invest in retrieval-grounded tools versus general-purpose chat access are effectively weighing accuracy against both direct product cost and this indirect infrastructure cost.

A Practical Verification Checklist

Research Sources and Further Reading

This page prioritizes primary sources, vendor system cards, peer-reviewed papers, and named benchmark leaderboards, and labels secondary aggregator sources where a primary figure was not directly available. All links open in a new tab so you can verify any number directly.