How to Fact-Check AI Writing Before You Publish

By Austin Little

A model will hand you a clean paragraph with a fake statistic, a real-sounding court case, and a link that 404s — then thank you for your time. Fact-checking AI writing is not optional polish. It is the job. This is the pre-publish checklist we use before anything leaves the hive.

AI disclosure. This page was drafted with AI assistance and edited for Apiary. We don't invent quotes, stats, people, or events. If something looks off, tell Austin — that's the point of a living hive.

The real problem is confidence, not spelling

Spellcheck catches typos. AI errors often look finished. The grammar is fine. The tone matches your outline. The only problem is that the mite treatment dose, the license dollar threshold, or the "study from 2019" never existed.

Treat every AI draft as a suspicious intern: fast, eager, sometimes brilliant, not allowed to publish alone.

Pair this page with AI Disclosure on Articles so readers know the process, and with How to Use AI Without Paying if you need a free local stack for private drafts.

A 20-minute pass that catches most disasters

You do not need a newsroom. You need a habit.

Minute 0–2: Scope the risk

Ask: if this page is wrong, who gets hurt?

• Low risk: taste, opinion, metaphors, obvious how-tos ("open Terminal").
• Medium: product feature lists, pricing vibes, "best free tools" roundups.
• High: medical, legal, financial, structural safety, pesticide/treatment doses, anything a reader might do with their body, money, or hive today.

High-risk pages get more time and primary sources. If you cannot verify, cut the claim or mark it clearly.

Minute 2–8: Harvest every checkable claim

Skim and highlight (or list):

• Numbers and percentages
• Dates and "as of" claims
• Names of people, companies, laws, papers
• Quotes
• "Experts say" / "studies show"
• Prices, limits, plan names
• Step-by-step commands that could brick something
• Links

If a sentence has none of these, it can wait. If it has three, it goes on the list first.

Minute 8–18: Verify against primary sources

For each claim:

1. Find the primary page (agency, docs, paper, company pricing, statute overview from the agency itself).
2. Confirm the claim still matches today.
3. If you cannot confirm in a few minutes, delete, hedge, or flag — do not "leave it for later" on a live URL.

Secondary blogs are clues, not proof. AI likes citing other AI-written blogs. That is a hall of mirrors.

Minute 18–20: Links and final read aloud

Click every link. Read the risky sections out loud. Out-loud reading catches fake smoothness — sentences that sound smart but say nothing, or say something you would never stake your name on.

The hall-of-mirrors problem

Models are trained on internet text. A wrong claim can appear in twenty places. Searching the claim and finding "lots of results" is not verification. You want the origin:

• Government or university extension page for bee/pest guidance
• Official docs for software
• Court or legislature primary text for legal thresholds (or an agency plain-language page)
• The company's own pricing/docs for product limits
• A paper's abstract/PDF for scientific claims — not a LinkedIn summary of a summary

If the only sources are SEO blogs with the same paragraph structure, assume contamination.

Quote rules (strict)

• Do not invent quotes. Ever.
• If the model outputs a quote, assume it is fake until you find the original.
• Paraphrase with a link instead of a decorative quote when you are unsure.
• For living people, prefer linking their words over "recreating" their voice.

Fake quotes are not a cute error. They are a reputation bomb.

Numbers rules

• Prefer ranges and hedges when the exact figure churns ("free tier limits change — check the provider page").
• Do not round in a way that changes meaning.
• Units matter: ml vs oz, °C vs °F, cycles vs years on a garage spring.

Product and free-tier claims

"Free AI" articles rot fast. Before publish:

• Open the provider's current free/pricing page yourself.
• Note what requires a card.
• Avoid "unlimited" unless the page literally says so and you believe it for the next month.
• Prefer evergreen wording: "check the free tier page; limits move."

Same for model names. llama3.2 today might be the wrong pull command next spring. Point people at the project's own library list.

Beekeeping and field-service special cases

Apiary sits next to real hives and real garage work. Extra rules:

• No treatment recipes from chat alone. Models invent concentrations.
• No "this is legal in your city" without a primary local source. HOA and municipal rules vary.
• No part numbers for springs/openers from memory. Wrong part is expensive and dangerous.
• Photos: do not let a model "identify" a disease from a blurry image as certainty. Say "possible concerns to discuss with a mentor," then go look in the box.

For learning bees with guardrails, see Can AI Help Me Learn Beekeeping?.

A printable checklist

• [ ] Risk level tagged (low / medium / high)
• [ ] All numbers listed and checked or removed
• [ ] All proper nouns checked
• [ ] All quotes verified or removed
• [ ] All links clicked (no 404, no unrelated page)
• [ ] Commands tried on a safe machine when non-trivial
• [ ] Disclosure block present
• [ ] "As of" date for churny sections
• [ ] Someone with domain taste read the high-risk parts (even if that someone is you tomorrow morning)

Workflow inside a BYO / local setup

Fact-checking is easier when drafts stay local until you are ready:

1. Draft in a local model (Ollama) so half-wrong pages are not sitting in a random cloud account.
2. Keep a claims.md scratch list beside the draft.
3. Verify in a normal browser with bookmarks to trusted primaries.
4. Only then paste into CMS / Apiary publish path.

BYO-LLM means the site does not own your brain; it does not mean the model owns your facts. See How Apiary BYO-LLM Works for Readers.

What "good enough" means for a small publisher

You will not match a metro newsroom on every post. Good enough means:

• No fabricated people, quotes, or studies
• High-risk advice hedged or sourced
• Links work
• Disclosure honest
• You would defend the page to a skeptical friend in your trade

Ship that. Iterate. Waiting for perfection is how useful pages never help bees or readers.

Common AI failure modes (with fixes)

Editing for voice after facts are sound

Fact-check first, voice second — or you will polish a lie until it shines. Once facts are solid:

• Cut empty openings ("In today's world…")
• Replace vague "many experts" with named sources or delete
• Keep Austin/Apiary plain speech: short sentences, concrete steps, no fake warmth
• Cross-link related hive pages instead of repeating a whole guide

For writing craft without sounding like a brochure, the sibling topic is How to Write With AI Without Sounding Fake (queued in craft-wave1) — until that ships, the rule is: your examples beat the model's stock anecdotes.

Team review without bureaucracy

Two roles beat twelve meetings:

1. Verifier — owns claims and links.
2. Reader — owns clarity and whether the page answers the search query.

On solo Apiary days, do verifier in the afternoon and reader the next morning. Sleep is a fact-checking tool.

When to kill the article

Delete or park as draft if:

• The query only works with live data you cannot maintain monthly.
• The topic is high-risk and you lack access to primaries.
• The model draft is mostly filler and you do not actually know the subject.

Empty calories with a disclosure still waste a reader's hour.

Joe-Google test

People type:

• "how to fact check AI writing"
• "ChatGPT wrong facts"
• "verify AI article before publishing"
• "AI hallucination examples"
• "edit AI content checklist"

Give them a checklist and failure table, not a philosophy seminar.

Practice drill (15 minutes)

Take any AI paragraph about a tool you use. List five claims. Verify each. You will usually find one soft invention. That discomfort is the skill forming.

Do the same later with a bee or garage paragraph without publishing — feel how much higher the bar is when wrong advice has physical consequences.

Cross-links

• Disclosure labels: AI Disclosure on Articles
• Local drafting: Free AI for Writing Blog Posts Locally
• Students: Best Free AI for Students — academic integrity is a cousin of fact-checking

Sources / further reading

• Apiary: https://apiarybee.com
• Related: How to Use AI Without Paying; What Is an AI Beekeeper?

Building a personal primary-source shelf

Speed comes from bookmarks, not from bravado. Keep a folder named primaries with links you actually use:

• Software: Ollama docs, provider pricing pages you cite, WebLLM notes
• Bees: your state extension apiculture pages, club handouts you trust, pesticide labels when relevant
• Business/local: CSLB or your state's contractor board, FTC consumer pages when you write scam-prevention
• Apiary product: https://apiarybee.com for BYO claims — do not freestyle features

When the model invents a URL, do not "fix the typo" into something that looks right. Go from your shelf or a careful search.

Version pinning for tutorials

Commands rot. When you publish install steps:

• Prefer official installers and docs links over mirrored blog steps
• Say "menu names move; trust the project's current README"
• Re-run the happy path on a clean machine when the article is a cornerstone

Emotional traps while checking

• Sunk cost: you spent an hour on the draft, so you keep a flaky claim. Cut it.
• Authority cosplay: the model sounded like NIH, so it feels true. Still check.
• Deadline theater: "we need something up." Wrong high-risk content is not content.
• Friend bias: a collaborator loves the paragraph. Love is not a source.

Logging corrections without shame

Keep a simple corrections.log: date, URL, what was wrong, what fixed it. After three months you will see patterns — maybe model names, maybe legal thresholds, maybe affiliate-shaped tool claims. Patterns tell you where to slow down next wave.

Fact-checking AI images and diagrams

If you publish diagrams the model generated:

• Read every label on the image
• Check anatomy / wiring / hive parts against a trusted diagram
• Do not let a pretty schematic teach a dangerous wiring move
• Caption AI images as illustrated, not photographed, when that could confuse

The "teach-back" test

Explain the risky section to an imaginary beginner in your own words without looking at the draft. If you cannot, you do not understand it well enough to publish under your name. Go learn, then rewrite. AI speed does not replace that loop.

Case study walkthrough (composite, not a real scandal dump)

Imagine a draft paragraph:

"According to a 2021 USDA report, backyard beekeepers who used Product X saw a 62% drop in varroa within 48 hours. Dr. Elena Marquez of the Bay Area Bee Institute called it 'the only treatment that always works.' CSLB also requires a license for any hive stand over $400 in California."

Fact-check pass:

1. USDA report + 62% + Product X: search USDA and the product. If no such report, delete the whole statistic. Do not "adjust to 40%" from vibes.
2. Dr. Elena Marquez / Bay Area Bee Institute: search. If the person or institute is invented, remove the quote entirely. Never keep a fake expert because the sentence sounds nice.
3. "Only treatment that always works": even a real person should not be paraphrased into absolute medical language without a source. Absolutes about pests are a red flag.
4. CSLB + hive stand + $400: CSLB is a contractors board; hive stands are usually not its lane. The dollar figure may be confused with handyperson exemptions. This claim is cross-domain nonsense — cut it. If you need contractor thresholds, cite CSLB primary pages in a garage article, not a bee treatment graph.

Composite lesson: one paragraph can contain four different lie types. Your checklist has to catch all four.

Research notes vs published voice

Keep two files:

• notes-raw.md — links, quotes with URLs, doubts, "AI suggested X — unchecked"
• article.md — only claims you are willing to stand behind

Never paste unchecked notes into the article file "temporarily." Temporary becomes permanent on launch day.

Automated helpers (use carefully)

Useful:

• Link checkers
• Spellcheck
• Your own script that lists URLs in a markdown file

Not sufficient:

• "AI fact check" buttons that only ask another model
• Plagiarism scores as a substitute for primary sources
• Detectors that claim to prove truthfulness

Automation can list candidates. Humans still verify.

Scheduling re-checks for living pages

Cornerstone craft pages need a calendar:

• Free-tier / model install pages: every 1–2 months
• Legal threshold mentions: when you hear of a law change, and at least yearly
• Seasonal bee pages: before the relevant season
• Product feature pages for Apiary: whenever the connect-brain UI changes

Put the next review date in the frontmatter or a queue comment. A hive that never re-opens old frames gets pests.

Teaching juniors or agents to check

If an agent drafts inside Apiary:

• Require a claims section in the PR or folder
• Score drafts on "primary links present," not on word count alone
• Reward deletions of shaky claims as much as new sections

Word count without verification is just a longer risk surface. This wave's BIGGIE length assumes substance — sources, steps, failure modes — not filler adjectives.

Reader-facing uncertainty language that still sounds adult

Prefer:

• "As of September 2026…"
• "Check the provider's current free-tier page."
• "Confirm with your club mentor before treating."
• "This is informational, not a diagnosis."

Avoid:

• "It is widely known that…" (by whom?)
• "Science has proven…" (which paper?)
• "Always do X" on biological systems that vary by climate

When your source disagrees with your lived experience

Say both. "Extension page says X; in Fremont yards I still see Y — here's how I reconcile." That is honest craft. Letting the model average the internet into a fake consensus is not.

Final gate before publish

Print or full-screen the article. Ask:

1. What is the single sentence a hurried reader will remember?
2. Is that sentence true?
3. If they only do one action from this page, is that action safe?

If (2) or (3) fails, you are not done.

Quick reference card (save this)

1. List claims. 2. Primary source each. 3. Click links. 4. Kill fake quotes. 5. Hedge churn. 6. Disclose. 7. Sleep once on high-risk. 8. Publish. 9. Re-check on a calendar.

That card is the whole craft. Everything else in this article is commentary so you do not skip a step when the draft flatters you.

FAQ

Can I ask the same model to fact-check itself? It can help spot weak spots. It cannot be the final authority. Use primary sources.

Are AI detectors useful here? Not for truth. Detectors guess authorship. You need accuracy.

How many sources per claim? One strong primary beats three blogs. Two primaries for contested claims.

What about common knowledge? "Honey bees are insects" does not need a footnote. "This treatment dose is safe" does.

Should I show my claim list publicly? Optional. Sources section is enough for most craft pages. Researchers may appreciate more.

Does local AI hallucinate less? It can still invent. Privacy is not truth. Same checklist.

Ship fewer pages. A wrong high-risk page costs more than a delayed one.

How do flags work in drafts? Do not publish raw flags without a decision.

Is hedging cowardly? Hedging uncertain live details is honest. Hedging everything to avoid checking is lazy. Know the difference.

What is the one rule if I forget the rest? No fake quotes, no fake studies, no unverified doses. Everything else is technique.

Deep dive: editing pass after the machine draft

When a local or free model returns a draft, do not publish the first paste. Run this pass every time:

1. Delete throat-clearing. Cut openings that begin with "In today's world," "In the digital age," or "As we all know."
2. Underline every number. Each underline needs a source or a cut.
3. Underline every proper noun. People, agencies, products, statutes — verify spelling and existence.
4. Strip absolute medical or legal commands. Replace with "ask a licensed professional / mentor / clerk."
5. Read one section aloud. If you stumble, the reader will too.
6. Check the Joe-Google promise. Does the first screenful answer the query that earned the click?
7. Add disclosure if AI helped in a material way.
8. Click every link. Dead links are a trust tax.

This pass is slower than generation. That is the point. Free AI shifts time from typing to verifying — not from work to zero work.

Deep dive: household policy sheet (one page)

Write these lines once and tape them inside a cabinet or password binder:

• Approved AI paths in this house: __________________
• Forbidden pastes: bank, SSN, Medicare, passwords, unfinished legal threats, other people's medical data
• Who to call when confused: __________________
• Paid plans: only with calendar cancel reminder
• Children: school policy overrides household curiosity
• Bees / garage / health actions: human experts before model advice

Households that skip this sheet re-litigate the same fight every holiday.

Deep dive: common failure stories (composites)

The subscription creep. A free trial for "research" becomes twelve months of a plan nobody opens. Fix: local default; calendar reminder on any trial day 0.

The confident wrong dose. A model mashed EU and US treatment guidance. Colony harmed. Fix: labels + mentor only.

The volunteer leak. A newsletter editor pasted a counseling blurb into a cloud chat. Fix: local drafts; privacy training.

The student shortcut. An essay submitted with invented citations. Honor board. Fix: quiz-from-my-notes only; verify every citation.

The five-app weekend. Grandma got Claude, ChatGPT, Gemini, Perplexity, and a random toolbar. She uses none. Fix: one path, sticky note.

These are teaching composites, not claims about named clients.

Deep dive: metrics that matter

Track for a month:

• Hours saved on outlining (honest guess)
• Number of factual corrections you made to model output
• Number of times you almost pasted a secret
• Whether anyone paid a surprise bill
• Whether a second human could repeat your setup from the sticky note alone

If corrections are zero, you are not looking hard enough. If surprise bills are non-zero, the free path failed operationally.

Deep dive: relating craft to bees and Apiary

Apiary's public stance ties knowledge work to living bees. Craft articles are not a detour — they are how volunteers, students, and families gain tools without extractive lock-in. BYO-LLM is the product expression of that ethic. Free-path teaching is the cultural expression. Disclosure and fact-checking are the editorial expression. When those three align, the hive metaphor is not branding cosplay.

Site: https://apiarybee.com

Use HTML comments in drafts:

Typical reasons: live pricing, model names, UI labels, legal thresholds, medical-adjacent wording, third-party endorsements. Agents do not invent the answer to clear the flag.

Deep dive: accessibility checklist for every craft page

• Plain language first; jargon defined on first use
• Short paragraphs
• Lists for procedures
• High-contrast assumptions (do not rely on color alone in diagrams)
• Avoid seizure-flashy GIF advice in tutorials
• Provide sticky-note / printable summaries for older adults
• Caption urgency without panic fonts

Deep dive: what "free-only advice" means operationally

When we say free-only AI advice in this wave, we mean:

• The recommended default path works at $0
• Paid tiers may be mentioned as optional horsepower
• We do not require a card to complete the tutorial
• We warn about trial-to-paid conversion
• We prefer local and in-browser on-device options when privacy matters

If a paragraph accidentally reads like an affiliate funnel, rewrite it.

Deep dive: after-publish stewardship

Day 0 publish is not the end. Put a reminder:

• 30 days: tool names still accurate?
• 90 days: free-tier still free without card?
• Seasonally: safety examples still responsible?
• When mail arrives from readers: log corrections publicly if the error mattered

A living hive edits. A brochure site pretends forever.

Mini FAQ add-on

Can I automate fact-checking fully? No. Can I skip disclosure if the AI only fixed commas? Usually a light note is enough; be consistent. Can I use AI to write apology letters after a mistake? Yes — then send as yourself. Can Apiary staff see my local Ollama chats? Not by design of local inference; protect your device. Is longer always better? No. This wave's floor exists because these topics need steps and guardrails — not because fluff is virtue.