You Can’t Prompt Your Way Out of AI Hallucination

When the article came back

An article came back from our writer carrying a clean, confident statistic — a percentage we had never given it. We called it an AI hallucination right away. The sentence read like this: “Nearly 42% of small publishers reported…” It looked right. It wasn’t in the facts we supplied at all.

We had a rule in the prompt: never invent a fact; if unsure, say you don’t know. That rule cut the made-up facts a lot. Still, a few slipped through. They were the smooth ones. A tidy number. A source name that sounded real. A compact statistic that completed the sentence. The words had the confident cadence editors like.

The scene where it flipped

We sent the writer a brief set of facts — a spreadsheet of exact numbers, quoted lines from interviews, and named reports we would accept. The writer ran a draft through our usual instructions, which included the strict “never invent” line. Draft looked clean. We read it. We marked edits. One paragraph contained that percentage. No corresponding cell in our spreadsheet. No quoted line in the facts.

We paused. Then we ran a second thing: an automated reader that checked every concrete claim against the fixed list of supplied facts and refused anything not on that list. It didn’t make suggestions. It refused. It flagged the paragraph and blocked publication. That was the turning point. We watched it refuse the confident number, and for the first time the check and the prompt visibly disagreed in public.

Before that, we treated the “never invent” instruction as the defense. We had meant it to be the gatekeeper. But the blocker — the reader that compares claims to supplied facts and refuses mismatches — became the defense we trusted. The moment it refused a draft that the prompt had “passed”, the rule changed for us: the instruction is the first layer, not the final arbiter.

What we tried, and what happened

We experimented in three clear moves. First, we tightened the prompt. We wrote a strict line: “Never invent a fact. If a fact is not in the supplied facts, say ‘unknown’ or leave it out.” That cut fabrication sharply. It did not eliminate it. We still saw confident-sounding fictions.

Second, we made the grounding explicit. Every claim in the draft had to cite which supplied fact it mapped to. We asked the writer to attach the exact row or quote. Drafts returned cleaner. The automated reader’s job became easier. Clean on the first try increased.

Third, we put the separate automated reader in front of publication. It compared each concrete claim to the fixed set of source facts we had given. Anything that couldn’t be matched was blocked. No negotiation. That is what actually stopped the fabrications cold.

Here is one representative detail of what we asked the writer to include: a single inline note after every data claim showing the exact source row number. That small, visible mapping made the automated reader’s life simple and ours clearer. I won’t paste the whole checklist here; the tested, runnable version lives in the members’ library.

How the discoveries fit together

We learned three things, from trials we can show.

First: strict prompt rules reduce mistakes. When we forced the writer to say “unknown” if unsure, the frequency of made-up facts dropped noticeably. Fewer obvious inventions appeared.

Second: the confident fictions persisted. The leftovers were always tidy-sounding items — a plausible percentage, a crisp citation. The writing voice filled blanks. The prompt alone couldn’t reach those; the instruction reduced them but didn’t stop them.

Third: a separate check that compares claims to a fixed list of facts is the stopper. When that check refused a confident but unsupported claim, it prevented publication. We watched it refuse. That refusal was the proof.

Rule that fell out: treat the instruction as the first layer, and trust the reader that refuses things. Only trust a check you have watched refuse something. We say this as a practice rule, not a slogan. It changed our workflow the day we saw a draft the prompt accepted and the reader rejected. The reader winning made the rule real.

One counterexample

We should be honest about a counter-case. There was a stretch where the automated reader blocked far more drafts than we expected. The prompt had been weak — vague about what counted as “a fact” — and the reader kept refusing anything that required even minor interpretation. That created a lot of rework. We tightened the prompt and clarified which kinds of claims the fixed facts covered. Then the reader’s refusals focused on real fabrications, not edge interpretations.

So the prompt still matters. It is not ceremonial. A weak prompt produces lots of useless drafts the reader must refuse. A better prompt saves wasted work because fewer drafts reach the block step. The prompt never replaced the reader, but good instructions reduced the number of times the reader had to refuse a draft.

Why this works

People and machines make confident errors for the same reason: the language wants to complete the sentence. The model fills that gap with plausible content. A strict instruction interrupts that habit but does not fully stop the urge to finish the thought. An automated reader that checks claims against a fixed list of facts enforces a harder constraint.

Think of the prompt as setting expectations. It nudges the writer to be careful. The automated reader is the referee with the scorecard. The referee can refuse points when there is no proof. Together they reduce AI hallucination much more than either could alone. We call that the two-step check in conversation, but the key is simple: a human-style rule followed by a mechanical proof-checker.

Questions we got

Can you prompt away AI hallucination?

No. A strict prompt helps a lot and reduces obvious inventions, but it does not stop every confident fabrication. What finally stopped the remaining ones was a second system that checked every concrete claim against our supplied facts and refused anything not backed.

Is the automated reader a cure-all?

It is not perfect. It blocks claims that require interpretation if the facts don’t cover subtle context. It also requires the facts to be well-structured. Still, when we watched it refuse fabricated claims, we trusted it more than any instruction alone.

How much effort does this add?

There is upfront work: the facts must be prepared and mapped. That work reduces downstream edits because drafts arrive cleaner. We keep the tested, runnable version in the members’ library for teams who want the exact config; here we only show the approach and one inline mapping detail.

How we know: these are practices we built into our content pipeline and watched over months. We recorded the draft versions, the refusals, and the edits. When the automated reader blocked a confident number the prompt had allowed, that moment changed our trust.

Stopping most invented facts is possible. Stopping every single one is a higher bar and requires the reader you have seen refuse a draft. The gap between reduction and elimination is real.

Sources: internal product testing and our own content pipeline experiments.