The Publishing-Gate Skill: Content That Has to Prove Itself

When our gate became a loadable skill, it stopped trusting itself

We turned the verified-publishing-gate that runs our pipeline into a loadable skill so other teams could reuse it. That’s when the surprise happened: an independent reviewer found two fatal gaps in version 1.0 — a checkpoint that referenced a value the procedure never produced, and a leak canary that was never actually planted. The checks existed on paper. They did not exist in practice. This broke the trust model hard and blocked a flagship draft from publishing until we fixed it.

What we tried, and where it tripped

We exported the gate as a reusable module and ran it in a staging run of our own content pipeline. The gate claims to do three things: collect every factual claim with provenance before writing, require a heavy grounding threshold, and run an independent judge plus a planted canary to detect context leakage.

Step by step we observed the flow. Claims were collected. The judge ran. Publication was attempted. But the verifier checkpoint referenced a value named in the code that never appeared earlier in the run. The canary string that should have been planted into the draft was absent. The judge could only check what was actually present — and so the soft checks produced a false pass. The staging run flagged the mismatch and the gate refused to let the draft go live.

The turning point — what changed at the moment it clicked

Independent review produced a concrete diff: the checkpoint code had an expectation for a generated fingerprint token that the content-authoring step didn’t return. The canary code path existed but was never invoked because the function that should plant the token had an early return on a different flag. We added two simple things to the loadable skill during that review: a mandatory return of the fingerprint token from the authoring step, and an explicit canary-plant call that runs before the judge evaluates the draft. After that change the same staging run showed the canary present and the checkpoint value populated. The judge could then actually fail correctly when we fed it a deliberately weak draft; the gate quarantined everything published since the last known-good sanity check, as designed. That was the moment the gate began to act like a gate, not a curtain.

The rule that fell out

If a gate checks for an artifact, the artifact must be produced earlier in the same run — and the check must verify presence end-to-end, not merely code-path existence. When we enforced that, the gate stopped producing false positives. When we didn’t, it offered the illusion of safety.

What this skill encodes (concrete, but not runnable)

• Claims-first publishing: every factual statement is recorded into a claim store with a source, a confidence label and a reference before writing begins — this puts sourcing ahead of prose.
• A grounding floor plus a verified-ratio guard: the gate blocks generation when source material is too thin, and requires at least 90% of claims verified with no unverified high-confidence claim before a draft may proceed.
• An independent judge that only blocks: the judge is separate from the writer’s model and it returns a hard block on failing verdicts; there’s no “publish with edits” downgrade.
• A planted canary to detect context leakage: a deliberately injected token appears in content and must be detected by the judge to validate isolation.
• Idempotent publishing by content fingerprint: publishing is keyed by a content fingerprint rather than byte equality, and every publish ends with a read-back — fetch the live page and verify status and fields against intent, failing loudly on mismatch.
• Quarantine on judge failure of a known-weak sanity test: if the judge approves a deliberately weak draft, the system quarantines every publish made since the last passed sanity check to guard against a judge that always says yes.

One representative detail: the read-back step is literal — after a publish call returns success, the system fetches the live page, compares the title, claim markers, and fingerprint, and fails the pipeline if anything differs. That read-back prevented silent drift in our staging run.

A fair counter-case

A gate is only as honest as its weakest check. Version 1.0 proved this. The design had checkpoints, but a checkpoint that references a missing value and a canary that was never planted gave a false sense of safety. The gate itself required a gate: independent review and a real-world staging run that tried to break it. Without those, you can ship a gate that polishes the paperwork while leaving the core failure modes open.

How we know this works here

We run the same pattern in our production pipeline and watched it block our flagship draft before publication. Logs show the judge failing the canary test and the quarantine routine rolling back the subsequent publishes to the last known-good checkpoint. The preflight read-back confirmed the published page matched intent after the fix. Those concrete runs and the review diff are the source of the story.

What it means for teams

Building a gate as a loadable skill exposes gaps you don’t notice when it lives in one codebase. Tests must produce the artifacts their checks expect. Independent review and staging runs are non-negotiable. And planting a canary plus doing a real read-back turns bright-sounding checks into actual protection.

The tested, runnable version of this skill lives in the members’ library.

FAQ

Why record claims before writing?

Because capturing source, confidence and a reference up front forces the writer to confront grounding choices before prose hides them; it turns implicit assumptions into explicit artifacts the judge can check.

What does a quarantine do, exactly?

When the independent judge approves what we know to be a weak draft, the system quarantines every publish done since the last passed sanity check so you can investigate whether the judge has been compromised; it prevents a runaway false-positive from propagating unnoticed.

Can this gate create false negatives — blocking legitimate drafts?

Yes. The grounding floor and the 90% verified ratio are strict; they will stop borderline but possibly ok drafts. That’s by design: the gate errs on blocking. Balance comes from tuning and deliberate canaries and from review of blocked cases.

Sources: internal verified-publishing-gate v1.1 specification, the staging and pipeline run logs that flagged version 1.0, and the QA review that produced the fix.