How to Fact-Check AI Citations Before You Share Them

By Austin Little

An AI tool can hand you a citation that looks perfect: real-sounding authors, a respectable journal, a year, even a page range. Sometimes it points to nothing. This is the check I run before I put my name next to any source a model gave me. It takes about ten minutes per claim, and the tools are free.

AI disclosure. This page was drafted with AI assistance and edited for Apiary. We don't invent quotes, stats, people, or events.

I write about bees, garage doors, and AI tools. In all three, people share things fast. A neighbor forwards a "study" about a pesticide. A client quotes a "safety standard." A friend pastes a chatbot answer with five neat references at the bottom. It looks finished. It isn't finished until somebody checks it.

This guide is for anyone who's about to share a source: in a newsletter, a club post, a school report, a work memo, or a comment. You don't need a library card or a paid database. You need a browser, a few free sites, and the habit of not trusting a reference until you've seen it with your own eyes.

Why AI citations go wrong

Language models write by predicting what text should come next. A citation has a very predictable shape: names, year, title, journal, volume, pages, a DOI or a link. A model can produce that shape very well even when there's no real paper behind it. It isn't lying the way a person lies. It's filling in a pattern.

That's why AI citation problems usually fall into a few types. Once you know them, they're easier to catch.

The five ways a citation fails

1. The paper doesn't exist. The authors might be real people, and the journal might be real, but that title was never published.
2. The paper exists, but it doesn't say that. The source is real, and the claim attached to it isn't in it, or is stretched past what it says.
3. The details are mixed up. Right paper, wrong year. Right title, wrong journal. Authors from one paper attached to the title of another.
4. The link is dead or wrong. The URL returns a 404, or it goes to a different page entirely, or to a search results page that proves nothing.
5. The paper was real and then retracted. It was published, and the publisher later withdrew it. Someone citing it today is citing something the journal no longer stands behind.

A good check catches all five. It doesn't take long once you've done it a few times.

Tools that search the web can still get this wrong

Some AI tools search the web and show links. That helps, but it doesn't remove the need to check. The link might be real while the summary of it is off. Or the tool might cite a page that cites another page, and the original claim is somewhere else. Treat linked answers as a head start, not a finished job.

The ten-minute citation pass

Here's the method. I run it on every source I plan to share. If I'm short on time, I run it on the one source that carries the most weight in what I'm saying.

Step 1: Copy the citation exactly, before you touch anything

Paste the full citation into a plain note: authors, year, title, journal or publisher, volume, issue, pages, DOI, link. Don't fix anything yet. If the AI gave you a slightly broken citation, you want a record of exactly what it said. That record helps you spot which parts were real and which were filled in.

Step 2: If there's a DOI, resolve it

A DOI is a Digital Object Identifier. The DOI Foundation describes it as a unique identifier made of a prefix and a suffix separated by a forward slash, and gives the example 10.1000/182. You can resolve one by putting it after https://doi.org/, so https://doi.org/10.1000/182.

What to check when you resolve a DOI:

• Does it go anywhere? If the DOI doesn't resolve, that's a strong warning sign. A typo is possible, so check the characters carefully, but don't assume it's a typo.
• Does it land on the same paper? This is the one people skip. A model can attach a real DOI from a different paper. Compare the title, authors, and year on the landing page to your note from Step 1. All of them.
• Is it the publisher's page? The DOI should take you to the publisher or a trusted repository, not a random blog.

If the DOI resolves to the right paper, you've cleared the first two failure types for existence. You still need to check what the paper says.

Step 3: If there's no DOI, search the title in quotes

Paste the exact title in quotation marks into a scholarly search tool. Free options:

• Crossref Metadata Search searches metadata for journal articles, books, standards, datasets, and more. It's good for confirming that a title, author list, and year belong together.
• Google Scholar is good for finding the paper and seeing where else it appears. Its help page notes that abstracts are freely available for most articles, and that results are sorted by relevance unless you choose "Sort by date" or "Since Year."
• PubMed is the place for anything medical or biological. It's an official U.S. government site (the address sits under nlm.nih.gov), and its help page also shows you how to export a correctly formatted citation with the Cite button.

If an exact-title search turns up nothing in all three, the paper very likely doesn't exist as cited. Try a shorter piece of the title once, in case the model paraphrased it. If that also comes up empty, mark the citation as unverified and don't share it.

Step 4: Match every field

Once you've found a real record, line it up against what the AI gave you:

• Title, word for word
• Every author name, in order
• Year
• Journal or publisher
• Volume, issue, pages

Small differences can be honest formatting. Big differences, like a different first author, a different journal, or a year off by a lot, mean the AI blended sources. Use the real record, not the AI's version.

Step 5: Read the part that supports the claim

This is the step that matters most and the one people skip most often. A real paper doesn't help you if it doesn't say what you're about to say it says.

• Read the abstract first. Does the paper study the thing the claim is about?
• Find the specific sentence, table, or figure that supports the claim. If you can't find it, you can't cite it for that claim.
• Check the scope. A study on one species, one region, one age group, or one lab setting doesn't automatically apply to everything. "In a lab study of X" is a different claim from "X causes Y."
• Look for hedging words in the original, like "may," "suggests," and "in this sample." If the paper hedges and the AI summary doesn't, the summary is overselling.

If the full text is behind a paywall, the abstract is sometimes enough to confirm the paper's topic, but often not enough to confirm a specific number. In that case, say less. "A study found..." is weaker than you might want, but it's honest. Don't quote a number you didn't see.

Step 6: Check for retractions and corrections

A paper can be real, correctly cited, and still retracted.

Crossref's documentation on Retraction Watch data says that in September 2023, Crossref acquired the Retraction Watch database and made it publicly available. It says the database is gathered from publisher websites and updated every working day by Retraction Watch, and that it also includes some expressions of concern and corrections, though those aren't as complete as the retractions.

Ways to check:

• Search the paper's title or DOI in the Retraction Watch database. The Retraction Watch user guide explains how.
• Look at the paper's own landing page. Publishers often put a retraction notice, a correction, or an "expression of concern" right at the top.
• If you're comfortable with a little tech, Crossref's documentation describes a REST API filter for retractions and a downloadable dataset you can clone with git.

If the paper was retracted, don't cite it as support. If it's important to mention, say clearly that it was retracted.

Step 7: Fix or drop the link

For web links (not papers), open every link yourself:

• Does it load? A 404 means the page is gone or never existed.
• Is it the right page? A link to a site's homepage, or to a search results page, doesn't support a specific claim.
• Does the page actually say the thing? Use your browser's find feature to look for the key phrase or number.
• Is it the original source? If a news story cites a report, go find the report.

If a page used to exist and doesn't now, the Wayback Machine, which Open Library's site lists as another Internet Archive project, may have a saved copy at web.archive.org. That can confirm a page was real at some point.

Step 8: For books, find the book

AI tools often cite books, sometimes with chapter titles that sound right and aren't. Free ways to check a book exists:

• Open Library, which describes itself as an initiative of the Internet Archive, has book records and sometimes borrowable copies.
• WorldCat lets you find items in libraries near you.
• Your local library's catalog, and the librarian behind the desk, who is often the fastest fact-checker in town.

Confirming a book exists is the easy part. Confirming that a specific page says a specific thing usually means getting the book. If you can't, don't attach a page number you haven't seen.

Step 9: Write down what you checked

Keep a simple log next to your draft:

• Citation as the AI gave it
• What you found (real or not, right or wrong details)
• The link or DOI you used to confirm it
• The exact sentence or figure that supports the claim
• Date checked

This takes a minute and saves you an hour later when someone asks "where did you get that?"

Step 10: Decide what to share

You have three honest choices for each source:

• Verified: share it, with the corrected citation.
• Partly verified: share a softer version of the claim that matches what you actually confirmed.
• Unverified: drop it. Don't share it "with a caveat." People skip caveats.

Red flags you can spot in seconds

Before you even start the pass, some citations announce themselves as risky.

• Too perfect a fit. A paper whose title restates your exact question, word for word, deserves extra suspicion.
• Round, quotable numbers with no table behind them. "Studies show 73% of..." with nothing you can find is a warning.
• A journal name you can't find. Search the journal name by itself. If it doesn't turn up as a real publication, nothing in it will.
• Real authors, odd topic. A well-known researcher listed on a paper far outside their field is a sign the model borrowed a name.
• Links with long, plausible paths that don't load. Models are good at making URLs that look right.
• No way to find it except through the AI. If the only place a source exists is the chat window, it doesn't exist.

Retraction Watch also keeps public lists, including a Hijacked Journal Checker and a list of papers and peer reviews with evidence of ChatGPT writing, both linked from its site navigation. They're worth a look if you're checking academic sources.

Asking the AI to help (carefully)

You can use an AI tool to make checking faster, as long as you don't use it to do the checking for you.

Prompts that help

• "List every factual claim in this answer as a numbered list, with the source you used for each. Mark any claim that has no source." This turns a paragraph into a checklist.
• "For each citation, give me the exact title in quotes and the DOI if there is one. If you're not certain a DOI is correct, say so." This gives you clean search strings.
• "Which of these claims would be most harmful if wrong?" This tells you where to spend your ten minutes first.

Prompts that don't help

• "Are these citations real?" A model can't confirm that from inside itself. It may say yes either way.
• "Give me a source for this." That's an invitation to produce a citation-shaped string. If you need a source, search for one yourself and then ask the AI to help you read it.

Use a local model for private drafts

If the document you're checking is private, like a work memo, a medical question, or a family matter, you can run a model on your own computer with a free tool like Ollama. That way the draft stays on your machine. The checking still happens in public databases, because that's where the truth lives.

A worked example (with a placeholder)

Here's the shape of a real check, with a made-up placeholder citation so I'm not inventing a fake study about anything real.

Say an AI answer includes:

Placeholder, A., & Example, B. (Year). "Title of a Study About Your Topic." Journal of Something, Vol(Issue), pages. doi:10.xxxx/xxxxx

Here's what I'd do:

1. Copy it into my notes exactly.
2. Resolve the DOI at doi.org. Say it lands on a paper with a different title and different authors. That's a blend. The DOI is real, but it belongs to something else.
3. Search the exact title in Crossref, Google Scholar, and, if it's health-related, PubMed. Say nothing comes up in any of them.
4. Search a shorter piece of the title. Still nothing.
5. Decision: unverified. I drop the citation. If the claim matters, I go looking for a real source myself, starting with the database that fits the topic, and I only make the claim if I find one that supports it.

That's the whole job. It took less time to do than it did to read.

Special cases

Health and medical claims

Use PubMed first. Look for the type of study: a single small study is weaker than a systematic review. Check the date, because medical guidance changes. And never change a medication, dose, or treatment because of a citation an AI gave you.

Legal claims and standards

AI tools can produce case names, statute numbers, and code sections that look right and aren't. Check them against the official government or court source, or the standards body that publishes the code.

Bees, pesticides, and local rules

For beekeeping and pesticide questions, look for your state's agriculture department or a university extension service, and read the product label itself for pesticide claims. A citation to "a study" about bees and a chemical is exactly the kind of thing that travels fast and gets distorted.

Quotes

If an AI gives you a quote from a person, assume it's not verbatim until you find the original speech, interview, or document. Paraphrased quotes in quotation marks are a common way misinformation spreads. If you can't find the original, don't use quotation marks.

Building the habit

The first few times, this feels slow. After a week, it's quick. A few habits keep it that way.

• Check before you write, not after. It's much easier to build a paragraph around a verified source than to rescue a paragraph whose source fell apart.
• Keep your log. A running file of sources you've already confirmed means you never check the same thing twice.
• Prefer fewer, better sources. One source you've read beats five you haven't.
• Say what you didn't check. "I couldn't confirm this one" is a sentence that builds trust.
• Correct out loud. If you share something and later find it was wrong, post the correction where you posted the original.

FAQ

How can I tell if a citation from ChatGPT or another AI is real?

Resolve the DOI at doi.org if there is one, and make sure it lands on the same title and authors. If there's no DOI, search the exact title in quotes in Crossref Metadata Search, Google Scholar, or PubMed. If you can't find it anywhere, treat it as not real.

The paper is real. Am I done?

No. Read the abstract and find the specific part that supports the claim. Then check for a retraction or correction. Real papers get cited for things they don't say all the time.

What if the full paper is paywalled?

Use the abstract to confirm the topic, and say less. Don't quote a number you didn't see. Libraries can often help you get a copy.

Can I just ask the AI whether its sources are real?

You can ask, but the answer doesn't prove anything. The checking has to happen in a real database or on the real page.

How do I check if a study was retracted?

Search the title or DOI in the Retraction Watch database, which Crossref says it acquired in September 2023 and made publicly available, and check the publisher's page for a notice.

What's the fastest version of this check if I only have two minutes?

Resolve the DOI or search the exact title, and confirm the title and first author match. That catches the worst problems. Then don't make the strongest version of the claim until you've done the rest.

The short version

Every AI citation is a lead, not a fact. Copy it exactly. Resolve the DOI or search the exact title. Match every field. Read the part that supports the claim. Check for a retraction. Open every link. Log what you did. Then share only what you confirmed, and say plainly what you couldn't.

It's ten minutes. Your name is worth ten minutes.