The Revision Skill Born From 13 Logic Bugs a Judge Missed

When approval looked like certainty

We built a skill that tightens a draft without changing what it says or how the author sounds. The writer never saw the source texts. They worked only from a brief distilled from four third‑party writing guides by a separate curator. We added house rules: receipts, read-back checks, and hard stops.

On paper it seemed airtight. An independent reviewer ran the approval form and gave top marks. The review passed.

Then the mandatory third checker did its job and flagged thirteen real logic errors. Thirteen. Two of those were critical.

The two failures that taught us more

First failure: an early exit. Version 1.0 had a path that could finish the job and deliver the revised text without ever running the meaning‑preservation check. The text went out looking fine. The meaning check was never executed. That slipped past the reviewer.

Second failure: the edit log. Version 1.0 sorted edits by impact. That made the final proof-of-done mathematically impossible — replaying the edits in that order could never rebuild the final text. No third party could verify the transformation byte for byte.

We fixed both the same day and shipped version 1.1.

What we changed and what happened next

We added a mandatory third checker whose single responsibility is hunting logic errors. It did what it was built to do. It found the thirteen problems and forced a change.

We removed the early exit. Now every run ends with a read-back. Each revised sentence is compared to its original by an external similarity check with fixed thresholds. If meaning drifts beyond the threshold, delivery is blocked outright. No warnings. No soft fails.

We stopped sorting edits by impact. Edits are stored in application order with sequence numbers so a stranger can replay them byte for byte. Checksums let a third party verify the entire bundle without asking us for secrets.

All thirteen issues were closed the same day in the 1.1 release.

What we learned in practice

Finding, shown. Proof, shown. Why it worked follows.

Finding one: a review form that looks good can still miss real logical failures. The reviewer approved the skill on form yet the third checker found thirteen errors afterwards. That disproved our assumption that independent review alone is sufficient.

Finding two: execution order matters for verifiability. Sorting by perceived impact broke replayability. Storing edits in the order they were applied restored the ability to reconstruct the final text exactly.

Finding three: a read-back that blocks on meaning drift stops bad outputs. When we compared each revised sentence to its source with fixed thresholds, the system could not deliver anything that had drifted past our definition of “same meaning.” That changed the failure mode from “silent corruption” to “overt block.”

We also had a protective detail that mattered. Because the writer never had access to source texts, literal copying was impossible by design. That removed one class of risk before any checks ran.

A counter-example we lived through

The independent reviewer thought everything was fine. Their approval showed on the sheet. Yet the checker still found serious logic errors. That one case forced us to accept that a human or automated approver checking only form fields can be misleadingly confident. Approval is not the same as verification.

How we know

We reproduced the problem locally. We enabled the early exit path and ran a batch of drafts. Final outputs looked plausibly correct but the meaning check never executed. Enabling the mandatory third checker then flagged thirteen distinct logic failures across that batch. Re-sorting the edit log to application order allowed a replay tool to rebuild the final files exactly and to compute checksums that matched what was delivered.

We patched the code. We re-ran the same drafts. The read-back blocked any case where the similarity check failed. The replay succeeded. The third checker reported zero of those previous logic failures after the fixes.

The runnable skill file itself lives in the members’ library.

Rule that fell out: never confuse approval with verifiable execution — make checks observable, replayable, and blocking.

Questions we got

How did the early exit hide the meaning check?

It was a code path that short-circuited the final step. Outputs were packaged as if complete. The packaging did not include evidence that the meaning-preservation check ran, so the delivery looked valid while the check had been skipped.

Could the similarity thresholds block valid edits?

Yes, if set poorly. Fixed thresholds forced us to tune conservatively. When an edit was stylistic but borderline, the system blocked it and required human review. That trade-off was acceptable to avoid silent meaning changes.

Does replayability add latency?

Some. Storing edits in application order with sequence numbers adds a verification step, and checksums take time. The delay was small compared with the risk of undetectable corruption, so we accepted it.

Sources: our production run and skill files from July 24, 2026, which document the version 1.0 failures and the version 1.1 fixes.