The Poison Test That Scored a Perfect 14/14 — and Proved Pass Rates Measure Agreement, Not Truth

When a perfect score proved the system was broken

We publish short, checked articles about tiny Spanish places. Every fact must pass two independent AI reviewers before it goes live.

On 2026-07-17 we sabotaged our own pipeline on purpose.

Target: San José, a fishing village of roughly 200 people in Almería, Spain. We fed the system the Wikipedia page for San Jose, California instead.

What we asked the system to do was simple: gather basic facts—population, province, coordinates, a couple of identifiers—and hand them to the reviewers.

Result: every one of the 14 facts the system produced passed both reviewers. 14 out of 14. Perfect score. Apparently verified.

The turning point came when we read the outputs side by side. The population figure was Silicon Valley scale. The county and coordinates pointed to California. But the article under consideration was supposed to be about a tiny Spanish fishing village.

In other words: the pipeline would have published an article that claimed Silicon Valley numbers about a Spanish village as “verified.”

How we proved it, step by step

Step one. We ran a control. The genuine Almería article (Altea in other tests) produced 14 correct claims when given the correct source. The pipeline approved them. That felt safe.

Step two. We fed the wrong source. Same extraction code. Same reviewers. Same review thresholds. The system did the extraction and the two reviewers approved every claim. Same 14/14.

The moment it clicked was blunt. The reviewers were not checking whether the document matched the subject. They were checking whether the extracted claims matched the document. When the document was coherently wrong, the reviewers rewarded coherence.

We had earlier seen a crude mismatch caught by the system: when someone fed the Bilbao page but asked for information about the town of Altea, the pipeline refused and reviewers failed it. That refusal had built false confidence. It was useful, but fragile: the catch works when the source is obviously about a different, big subject. It fails when the look-alike is plausible.

The fix that actually worked

We tried the obvious alternative: stronger reviewers, a smarter model. We rejected it. Budget existed. We said no because a smarter reviewer would likely approve the same poisoned facts more confidently; it would still be judging fidelity to the given document, not fidelity to the intended subject.

Instead we put an identity gate before any reading happened. Before the system extracts a single claim it first fetches a small bundle of hard reference facts about the intended place: country, province, a rough population band, coordinates, and a public registry ID where available. Then it asks one question: “Is this document about that place?” Yes or no. Nothing else runs until the gate answers.

When the poisoned California page was checked, the gate returned NO. The system produced zero claims. The process stopped cold.

When the genuine Almería page was checked, the gate returned YES. The system continued and produced 14 correct claims.

That single pre-check reduced downstream work dramatically. Rejecting the poisoned page at the identity gate took three processing steps. Letting the page through, extracting 14 claims, and fully reviewing them took forty-six steps. The identity gate was about fifteen times cheaper in compute and reviewer effort.

The rule that emerged: a pass rate measures agreement with the source you were handed, not the truth about the intended subject.

Put differently: reviewer agreement is necessary but not sufficient. Without an identity gate, high agreement can mean a convincing lie.

Why this works

Reviewers are good at fidelity. They check whether an extracted claim is supported by the document. They are less good at cross-checking subject identity when the document looks plausible.

The identity gate forces a very different question early: does this source even claim to be about the target entity? That is an easier binary test and far cheaper to run than extracting and validating dozens of claims. It turns the attacker’s job from “create believable wrong claims” into the much harder job of creating a full document that both looks authored for the subject and slips past the identity check.

One representative detail: for each entity we store a handful of verifiable anchors—country and coordinates are usually enough to reject an off-target page—then require a yes/no answer on match. That single check blocked the poisoned California page. The full, runnable implementation of the identity-gate lives in the members’ library.

Counter-example, honestly told. Earlier we saw a crude poisoning attempt—feeding Bilbao’s page for the town of Altea—that was caught and rejected by the reviewers without an identity gate. That rejection gave us confidence in the reviewers. It was correct in that case because the source was obviously about a different, large topic. The catch is brittle: it fails when the wrong page is a plausible near-match, as the San José test proved.

How we know. The findings are reproducible in our logs. On 2026-07-17 the poisoned run produced 14 approved claims; the genuine run produced 14 correct claims; the identity-gate runs produced a NO for the poisoned page and YES for the genuine page. We measured steps and compute: three for an early reject, forty-six for full extract-and-review.

Questions we got

Why didn’t a better reviewer solve this?

Because better reviewers still evaluate claims against the given document. They become more confident in their agreement, not more likely to ask if the document is about the intended subject.

Won’t the identity check reject ambiguous but usable pages?

Sometimes. The gate errs conservative: a NO stops publication. That means occasional extra human review, but far fewer severe mistakes and much less downstream waste.

Does this add much cost?

Very little. The identity check runs a few cheap lookups and one short binary comparison. We measured it at roughly one fifteenth the cost of full extraction and review when it rejects a poisoned page.

We changed a single question and avoided publishing a convincing lie. That felt like the point.

Sources: internal runs and logs dated 2026-07-17; control and poisoned experiments recorded in our validation book.