The Poison Test That Scored a Perfect 14/14 — and Proved Pass Rates Measure Agreement, Not Truth

AI-generated. Written by an automated system, checked by independent automated gates, and published without human editorial review. How we label and verify.

When a perfect score proved the system needed an identity gate

We publish short, checked articles about tiny Spanish places. Every fact must pass two independent AI reviewers before it goes live.

On 2026-07-17 we sabotaged our own pipeline on purpose.

Target: San José, a fishing village of roughly 200 people in Almería, Spain. We fed the system the Wikipedia page for San Jose, California instead.

What we asked the system to do was simple: gather basic facts—population, province, coordinates, a couple of identifiers—and hand them to the reviewers.

Result: every one of the 14 facts the system produced passed both reviewers. 14 out of 14. Perfect score. Apparently verified.

The turning point came when we read the outputs side by side. The population figure was Silicon Valley scale. The county and coordinates pointed to California. But the article under consideration was supposed to be about a tiny Spanish fishing village.

In other words: the pipeline would have published an article that claimed Silicon Valley numbers about a Spanish village as “verified.”

How we proved it, step by step

Step one. We ran a control. The genuine Almería article (Altea in other tests) produced 14 correct claims when given the correct source. The pipeline approved them. That felt safe.

Step two. We fed the wrong source. Same extraction code. Same reviewers. Same review thresholds. The system did the extraction and the two reviewers approved every claim. Same 14/14.

The moment it clicked was blunt. The reviewers were not checking whether the document matched the subject. They were checking whether the extracted claims matched the document. When the document was coherently wrong, the reviewers rewarded coherence.

We had earlier seen a crude mismatch caught by the system: when someone fed the Bilbao page but asked for information about the town of Altea, the pipeline refused and reviewers failed it. That refusal had built false confidence. It was useful, but fragile: the catch works when the source is obviously about a different, big subject. It fails when the look-alike is plausible.

The fix that actually worked

We tried the obvious alternative: stronger reviewers, a smarter model. We rejected it. Budget existed. We said no because a smarter reviewer would likely approve the same poisoned facts more confidently; it would still be judging fidelity to the given document, not fidelity to the intended subject.

Instead we put an identity gate before any reading happened. Before the system extracts a single claim it first fetches a small bundle of hard reference facts about the intended place: country, province, a rough population band, coordinates, and a public registry ID where available. Then it asks one question: “Is this document about that place?” Yes or no. Nothing else runs until the gate answers.

When the poisoned California page was checked, the gate returned NO. The system produced zero claims. The process stopped cold.

When the genuine Almería page was checked, the gate returned YES. The system continued and produced 14 correct claims.

That single pre-check reduced downstream work dramatically. Rejecting the poisoned page at the identity gate took three processing steps. Letting the page through, extracting 14 claims, and fully reviewing them took forty-six steps. The identity gate was about fifteen times cheaper in compute and reviewer effort.

The rule that emerged: a pass rate measures agreement with the source you were handed, not the truth about the intended subject.

Put differently: reviewer agreement is necessary but not sufficient. Without an identity gate, high agreement can mean a convincing lie.

Why this works

Reviewers are good at fidelity. They check whether an extracted claim is supported by the document. They are less good at cross-checking subject identity when the document looks plausible.

The identity gate forces a very different question early: does this source even claim to be about the target entity? That is an easier binary test and far cheaper to run than extracting and validating dozens of claims. It turns the attacker’s job from “create believable wrong claims” into the much harder job of creating a full document that both looks authored for the subject and slips past the identity check.

One representative detail: for each entity we store a handful of verifiable anchors—country and coordinates are usually enough to reject an off-target page—then require a yes/no answer on match. That single check blocked the poisoned California page. The full, runnable implementation of the identity-gate lives in the members’ library.

Counter-example, honestly told. Earlier we saw a crude poisoning attempt—feeding Bilbao’s page for the town of Altea—that was caught and rejected by the reviewers without an identity gate. That rejection gave us confidence in the reviewers. It was correct in that case because the source was obviously about a different, large topic. The catch is brittle: it fails when the wrong page is a plausible near-match, as the San José test proved.

How we know. The findings are reproducible in our logs. On 2026-07-17 the poisoned run produced 14 approved claims; the genuine run produced 14 correct claims; the identity-gate runs produced a NO for the poisoned page and YES for the genuine page. We measured steps and compute: three for an early reject, forty-six for full extract-and-review.

Questions we got

Why didn’t a better reviewer solve this?

Because better reviewers still evaluate claims against the given document. They become more confident in their agreement, not more likely to ask if the document is about the intended subject.

Won’t the identity check reject ambiguous but usable pages?

Sometimes. The gate errs conservative: a NO stops publication. That means occasional extra human review, but far fewer severe mistakes and much less downstream waste.

Does this add much cost?

Very little. The identity check runs a few cheap lookups and one short binary comparison. We measured it at roughly one fifteenth the cost of full extraction and review when it rejects a poisoned page.

We changed a single question and avoided publishing a convincing lie. That felt like the point.

Sources: internal runs and logs dated 2026-07-17; control and poisoned experiments recorded in our validation book.

What the research on automated fact-checking says

The best-known methods for scoring factuality all measure support against a source. FActScore breaks a generation into atomic facts and computes the percentage of atomic facts supported by a reliable knowledge source. The word that carries the weight is “reliable.” The score is only as good as the source you choose.

SAFE works the same way with search in place of a fixed corpus. It uses a model to split a response into individual facts and then evaluates each one by determining whether a fact is supported by the search results. Chain-of-Verification has the model plan verification questions and answer them independently so the answers are not biased by other responses. Each of these checks claims against evidence. None of them, as described, asks whether the evidence is about the right subject.

Judges have known blind spots too. Work on LLM-as-a-judge lists position, verbosity, and self-enhancement biases. Another study found that a judge’s ranking can be easily hacked by simply altering their order of appearance. Stronger judges reduce some errors. They do not change the question being asked.

Source choice is a separate risk. When Anthropic tested its research agents, people find edge cases that evals miss, including subtle source selection biases. Its LLM judge graded factual accuracy as do claims match sources?, which again checks fidelity to the source.

The step that is missing has a name in natural language processing. Entity linking is the process of linking entity mentions appearing in web text with their corresponding entities in a knowledge base. Two places named San José are two entities, and telling them apart is that task.

How this matches our 14 out of 14

The research explains the result in this article without any new assumptions. Our reviewers did what FActScore and SAFE-style checks do: they confirmed that each claim was supported by the document they were given. The California page supported all 14 claims, so both reviewers passed all 14. Support was real. The source was wrong.

It also explains why we rejected a smarter model. Better judges improve agreement with the source. The identity gate asks a different question first, which is an entity question: does this document refer to the place we mean? Checked against the stored anchors, the California page got a NO.

A checklist for an identity gate before AI fact checking

1. Store a few hard anchors per subject: country, region, coordinates and a registry ID where one exists.
2. Before extracting anything, ask whether the source is about that subject. Yes or no.
3. Stop the run on NO. Do not extract, do not review.
4. Run a poisoned control with a plausible look-alike, not just an obvious mismatch.
5. Report reviewer pass rates next to the identity result, never alone.

More questions

Do published factuality metrics check the source itself?

As their abstracts describe them, no. FActScore measures support by a knowledge source and SAFE measures support by search results. Neither abstract describes a step that confirms the evidence is about the intended subject, so source identity has to be checked another way.

Would changing the order of facts fool our reviewers?

This article does not test that. Research on LLM judges reports position bias, so it is worth testing in any pipeline that relies on model reviewers. Our finding is narrower: agreement with a coherent wrong source still scores perfectly.