Can you prompt away AI hallucination? We caught one that looked correct.
An editor handed back an article with a neat statistic: “43% of managers do X.” It read like journalism. It quoted no source. That percentage wasn’t in the fact file we had given the writer. It wasn’t anywhere in our sources. It was invented inside the draft, and it sounded certain.
The scene
We had given the writer a strict instruction: “Never invent a fact. If unsure, say ‘unknown’.” The draft followed the voice and structure we wanted. Except for that confident number. The sentence wanted a number and the model supplied one. Clean. Convincing. Wrong.
We flagged it. The writer rewrote. The number disappeared. We asked: how did it happen? And more importantly, how do we stop it happening at scale, consistently?
What we tried, in the order we tried it
First, we tightened the prompt. We made the “never invent” line harsher. We added: every claim must map to a supplied fact. We asked the writer to append a source tag for each sentence. The drafts improved. Fewer inventions. Fewer guessy statistics.
Still—some fabrications survived. They were always the same kind:
- a plausible percentage inserted where the sentence expected a number,
- a tidy statistic attached to a non-existent study, or
- a confident-sounding attribution like “a recent survey found” with no survey in our facts.
They slipped past the prompt because they sounded right. The model obeyed the rule to “write clearly” and “be decisive.” That confident voice masked the fact it had made something up.
The turning point
We stopped treating the prompt as the final defense. Instead we used the prompt as the first step: strict instructions to reduce waste and steer tone. Then we added an automated reader that did a different job. It didn’t try to be persuasive. It checked every concrete claim against the exact list of facts we had supplied. If a sentence said “43%” and that percentage wasn’t in the fact list, the reader flagged or blocked the sentence.
The moment it clicked was obvious. We fed a batch of drafts with the tightened prompt into that reader. Drafts the prompt had softened still failed the reader when they contained invented claims. The reader refused them. Where the prompt alone had produced a handful of confident fabrications, the reader stopped them cold. We watched a draft go from accepted-looking to rejected with the exact offending sentence highlighted. That visible refusal made the difference: our trust moved from “the instructions will prevent all hallucination” to “this tool will refuse anything not backed by facts.”
The rule that fell out
Trust the reader that refuses things. Treat the prompt as the first layer: make it sharp enough to reduce obvious noise and save time, but don’t assume it will stop every made-up fact. Then run an automated check that compares every concrete claim to the fixed list of supplied facts and blocks anything not on that list. You must see it refuse something before you believe it.
Three first-hand findings we proved
1) A strong “never invent” instruction reduces fabrications but doesn’t eliminate them. We wrote a strict rule (“never invent; say ‘unknown’ if unsure”) and saw fewer made-up facts. Some still appeared—always the confident-sounding ones that fit the sentence’s expectations.
2) Making the requirement explicit — forcing every claim to map to a supplied fact — improved first-draft quality. When writers had to point each claim to a fact we gave them, the drafts that reached the reader were cleaner and needed fewer rewrites.
3) The thing that actually stopped fabrications was an automated reader that compared each concrete claim to the fixed set of supplied facts and refused any claim not backed by those facts. That refusal is what prevented confident hallucinations from slipping through.
A fair counter-example
We learned the prompt still matters. When the prompt was weak, the automated reader flagged far more drafts for refusal. A weak instruction meant the model produced drafts with lots of unsupported claims, which created extra work for whoever had to fix them. So a good prompt saved time by reducing the number of refusals, even though it never replaced the reader’s blocking role.
How we proved it, step by step
We ran four passes on the same set of articles.
Pass one: original prompt, no automated reader. Result: many confident fabrications; editorial fixes after the fact.
Pass two: tightened prompt (“never invent,” map claims to facts). Result: fewer fabrications, but a few confident ones remained.
Pass three: tightened prompt plus explicit requirement that each sentence include a source tag linking to our fact list. Result: cleaner first drafts; fewer corrections needed.
Pass four: tightened prompt, source tags, and the automated reader that refused any claim not in our fact list. Result: those confident, invented statistics stopped appearing in accepted drafts. The reader flagged them and refused publication until they were removed or matched to a real fact.
Why the automated reader works
Because the reader doesn’t try to be persuasive. It only checks: “Is this concrete claim present in our supplied facts?” If no, it refuses. It doesn’t matter how convincing the language is. That prevents the model’s tendency to fill gaps with plausible-sounding but false details.
Limits and trade-offs
The reader will block anything that isn’t explicitly in the fact set. That reduces hallucinations but increases rework when your fact file is incomplete. You can choose to expand the fact set, or to allow explicit “unknown” statements, depending on the article’s goals. We found that a tighter prompt plus the reader gave the best balance between quality and throughput.
FAQ
Can a prompt alone stop AI from making up facts?
No. A strong prompt reduces made-up facts and saves editing time, but it doesn’t drive fabrications to zero. Confident-sounding inventions still get through because the model favors fluent, decisive language.
What does the automated reader actually do?
It compares every concrete claim in the draft to the exact list of facts you supplied. If a claim isn’t matched to one of those facts, the reader flags or blocks it. In other words it refuses unsupported facts instead of trusting a prompt to prevent them.
Won’t this create too much extra work if our fact list isn’t complete?
Yes, at first. You’ll get more refusals until the fact list matches your content needs. But this is the point: it’s better to expand your facts deliberately than to let the system invent plausible but false details. A better prompt reduces the number of refusals, and a better fact file reduces the friction overall.
How we know: everything above is built from our own content pipeline experiments. We tightened prompts, watched what survived, added the per-claim mapping, and then introduced the automated reader that refused unmatched facts. Watching a draft go from accepted to explicitly refused at the offending sentence was the moment we stopped arguing about whether prompts alone were enough.
Rule in one line: make the prompt the first line of defense and the automated reader the one you trust to refuse anything unsupported.
Sources: ebizapple first-hand testing and editorial experience.
Leave a Reply