We almost published praise for our own tool. An automated check stopped it.
It looked fine on the screen. The headline was tidy. The writer was pleased. The piece quietly flattered a tool we ship. It never made it to the site because a second automated reader refused it.
The scene
We had a draft that read as friendly explanation. It also slipped into sales. That slip was subtle. It was a tone problem, not a factual error. The author, tired after a long sprint, read it and approved it. Then the independent check flagged it and refused to sign off.
What we tried and what happened
We had already built a simple rule into our workflow: nothing goes live until an independent check signs off, and the writer never gets to approve their own work. The ‘check’ is a second automated reader whose only job is to approve or refuse, and that never writes anything itself. It looks for things we tell it to enforce: tone boundaries, undisclosed promotion, and plain clarity.
We ran the usual publish flow. Draft written. Editor reviewed. Writer marked ready. The automated reader ran. It refused the draft because the tone had drifted into selling instead of explaining. The editor looked at the exact lines the reader flagged. We saw how a few turns of phrase shifted the piece from impartial to promotional. Removing two paragraphs and tightening a headline dropped the ‘sell’ score below the threshold. The check then approved the revised draft. We didn’t publish the original. We published the revised one.
The moment that changed our thinking
We had tests before, but the turning point was deciding to trust only a check we had watched refuse something. Trust in theory is cheap; trust after a concrete refusal is weighty. Once we saw the reader actually say no—and saw the change needed to make it say yes—we stopped treating the gate as ceremonial. A gate that never says no is not a gate.
The rule that fell out
Nothing in our own name goes live without an independent sign-off.
Three concrete findings from running this
1) The gate catches tone, not just facts. In this near-miss the automated reader didn’t fail the article for accuracy. It failed it for tone: the language had drifted toward selling instead of explaining, and we had not noticed until the check refused it.
2) Most drafts pass. The gate isn’t there to block the good ones; it’s there for the handful that look fine and are not—exactly the ones a tired person waves through. In our experience most submissions sail. A few that look fine fail in ways humans routinely miss.
3) The gate costs something. It will reject drafts that were probably fine and it adds a step to publishing. We kept it anyway because the cost of a single bad piece going live under our name was higher than the operational friction.
A fair counter-example
We learned the downsides the hard way. Sometimes the reader refused a draft that an experienced editor thought acceptable. We spent extra time arguing over borderline cases. That overhead is real. It slowed some launches and frustrated contributors. Still, when the reader stopped a piece that would have subtly promoted our product, the consequence made the extra step worth it.
How we proved the gate worked
We watched the refusal happen. The automated reader produced a highlight—specific sentences and a short label, ‘promotional tone.’ We edited those lines. The check re-ran and changed its decision. The moment the status flipped from refused to approved is when the policy stopped being theoretical. That one observed rejection is why we now refuse to publish without the independent sign-off.
Short practical note for non-technical teams
Call the second system what it is: a second automated reader whose only job is to approve or refuse. It doesn’t write content. It enforces a few clear rules (tone, undisclosed promotion, basic clarity). Build one, and then make sure it actually says no at least once. If it never refuses anything, either your thresholds are wrong or the gate is just theatre.
FAQ
Why use an automated checker instead of a human reviewer?
Humans miss subtle drift, especially when tired or invested. The automated reader enforces consistent rules every time. It doesn’t replace human judgment; it provides a consistent safety net that catches the drafts humans are most likely to wave through.
Won’t this slow publishing and frustrate authors?
Yes, sometimes. It adds a step and sometimes flags drafts that are arguably fine. That’s the cost. We accepted that cost because a single piece that looks fine but subtly promotes our product would do more reputational damage than the occasional delay or annoyance.
How do you decide what the reader checks for?
Keep it narrow. We focus on tone (undisclosed promotion), basic factual consistency, and clarity. Narrow rules produce useful, explainable refusals you can act on quickly.
Close with one clear thought: reading fine is not the same as being right.
Sources: ebizapple first-hand observations and product testing in our content pipeline.
Leave a Reply