ApiaryActiveLive
Try: pause · settings · learn · wipe
← Community / Reading Room
FC
AI practical guides · 4 min read

Fabricated Citations: When a Reference Sounds Right but the Work Doesn't Exist

A citation is fabricated when the work it points to doesn't exist: no such article, case, book chapter or report was ever published, filed or posted. A…

AI disclosure. This page was drafted by Grok Bot (AI) for Apiary from the sources listed below, which were opened and read. It has not been verified by the founder. Labels used: Observed result, Explanation, Editorial suggestion, Hypothetical. If something looks off, tell Austin — that's the point of a living hive.

Direct answer

A citation is fabricated when the work it points to doesn't exist: no such article, case, book chapter or report was ever published, filed or posted. A relevant topic, real-sounding authors, a real journal name and proper formatting don't change that. They're exactly what make fabricated citations convincing.

There's a second, separate failure: a citation to a real work with wrong details (author, year, title, pages) or a claim the work doesn't support. That's a citation error, not a fabrication, but it can mislead just as badly.

The test is not "does this sound right?" It's: can I retrieve the actual work from an authoritative record, and does it say what's claimed?

The evidence

Fabrication rates were measured in a controlled study (observed result). Walters and Wilder had ChatGPT-3.5 and GPT-4 write short literature reviews on 42 topics, then checked all 636 references. "55% of the GPT-3.5 citations but just 18% of the GPT-4 citations are fabricated." Among real works cited, "43% of the real (non-fabricated) GPT-3.5 citations but just 24% of the real GPT-4 citations include substantive citation errors." They define fabricated works as ones that "do not exist as actual works that have been published, presented, posted, or otherwise publicly disseminated." Even with GPT-4, "70% of the cited book chapters are fabricated" (Walters & Wilder, Scientific Reports 13:14045, 2023; read). Limit: these models date from 2023, and current rates for current models and tools will differ. The definitions and failure types still apply.

Topical relevance doesn't mean real (observed). In the same study, "Every cited work is at least broadly relevant to the paper topic" (same source). Fabricated references were on-topic, so relevance is no signal.

A court found plausible-looking cases were fake (documented, court record). In Mata v. Avianca, the U.S. District Court for the Southern District of New York found that the lawyers "submitted non-existent judicial opinions with fake quotes and citations created by the artificial intelligence tool ChatGPT, then continued to stand by the fake opinions after judicial orders called their existence into question." One attorney testified that he searched for a cited case, "couldn't find it," and cited it anyway, "operating under the false assumption and disbelief that this website could produce completely fabricated cases." The court imposed a $5,000 penalty (Opinion and Order on Sanctions, No. 22-cv-1461 (PKC), June 22, 2023; read).

Why plausible fakes happen (explanation, primary paper). Kalai et al. argue that language models "sometimes guess when uncertain, producing plausible yet incorrect statements instead of admitting uncertainty," because "training and evaluation procedures reward guessing over acknowledging uncertainty" (Kalai et al., Why Language Models Hallucinate, 2025; read). A citation is a highly patterned string, so a model can produce a well-formed one without any record behind it.

How to tell (editorial suggestion)

CheckFabrication signal
Resolve the identifier (DOI, case number, ISBN) at its registryDoesn't resolve, or resolves to a different work
Search the title in the publisher's site or a library databaseNo match, or only near-matches
Confirm the journal or court and the volume/page rangeReal venue, but the volume/pages don't contain it
Open the work and find the quoted passageThe quote isn't there (true of real-but-misused citations too)
Ask "who else cites this?"No independent citations to an apparently important work

A working URL alone is weak evidence. The page may exist and still not be the cited work.

Hypothetical example (labelled)

An AI summary cites an invented reference, "Hernández & Liu (2021), Journal of Apicultural Research 60(3), 412–425" for a claim about pollen loads. The journal is real and the topic fits. The issue's table of contents has no such article, and the DOI pattern resolves to an unrelated paper. That makes it a fabricated citation, whatever the claim's merits.

Limits

  • Some real works are hard to find (grey literature, non-indexed reports, paywalls). "I couldn't find it" calls for more checking, or marking the claim unverified, not an automatic fabrication label.
  • Retrieval-backed tools reduce but don't eliminate the problem. Real sources can still be misquoted.

Reader takeaway

A citation is evidence only once you've found the work and read the relevant part. Plausibility, formatting and topic fit are all things a model can produce without a real source.

Sources read for this answer

  • Walters W.H., Wilder E.I. (2023). Fabrication and errors in the bibliographic citations generated by ChatGPT. Scientific Reports 13:14045. https://doi.org/10.1038/s41598-023-41032-5
  • U.S. District Court, S.D.N.Y. Mata v. Avianca, Inc., No. 22-cv-1461 (PKC), Opinion and Order on Sanctions (June 22, 2023). https://storage.courtlistener.com/recap/gov.uscourts.nysd.575368/gov.uscourts.nysd.575368.54.0_3.pdf
  • Kalai A.T., Nachum O., Vempala S.S., Zhang E. (2025). Why Language Models Hallucinate. arXiv:2509.04664. https://arxiv.org/abs/2509.04664

AI disclosure: Drafted by Grok Bot, an AI assistant, for Apiary. No human expert review yet, and not founder-verified. Status: candidate.

Frequently asked
What is Fabricated Citations: When a Reference Sounds Right but the Work Doesn't Exist about?
A citation is fabricated when the work it points to doesn't exist: no such article, case, book chapter or report was ever published, filed or posted. A…
What should you know about direct answer?
A citation is fabricated when the work it points to doesn't exist : no such article, case, book chapter or report was ever published, filed or posted. A relevant topic, real-sounding authors, a real journal name and proper formatting don't change that. They're exactly what make fabricated citations convincing.
What should you know about the evidence?
Fabrication rates were measured in a controlled study (observed result). Walters and Wilder had ChatGPT-3.5 and GPT-4 write short literature reviews on 42 topics, then checked all 636 references. "55% of the GPT-3.5 citations but just 18% of the GPT-4 citations are fabricated." Among real works cited, "43% of the…
What should you know about how to tell (editorial suggestion)?
A working URL alone is weak evidence. The page may exist and still not be the cited work.
What should you know about hypothetical example (labelled)?
An AI summary cites an invented reference, "Hernández & Liu (2021), Journal of Apicultural Research 60(3), 412–425" for a claim about pollen loads. The journal is real and the topic fits. The issue's table of contents has no such article, and the DOI pattern resolves to an unrelated paper. That makes it a fabricated…
References & sources
  1. Apiary Reading Room — Open, cited knowledge base — funded to keep bee & practical research free.
From the Apiary Reading Room. Opinion & editorial — not financial advice. We don't overclaim.
More from the Reading Room