AI-generated. Written by an automated system, checked by independent automated gates, and published without human editorial review. How we label and verify.
Ninety days after the last of 270 guides went live, we opened the search data expecting a curve and got a single number instead: four clicks. Roughly 4,400 impressions, average position near 40, four clicks. The run had taken twenty days and about 19,000 credits at roughly 70 credits per guide, and that was the whole measured AI content ROI of it.
Nothing was flashing red. Every row in the queue had reached PUBLISHED. Every judge gate had returned PASS. Every scorecard was green. The board we had built to tell us whether the factory was working said, correctly, that the factory was working.
What we checked first, and why none of it found anything
We assumed something had broken quietly, because that is usually the answer: thin output, near-duplicate pages, an indexing problem.
The duplication theory died first. We compared 354 posts pairwise on eight-gram overlap; every pair came back below 0.10. The pages were not variations of each other. Then we did the arithmetic we should have done at the start. At average position 40, a click-through rate of 0.1 to 0.2 percent on 4,400 impressions predicts four to nine clicks. We got four.
That is the uncomfortable part. Four clicks was not a symptom of failure. It was the pages earning precisely what their position entitled them to. There was no bug to bisect and no gate to tighten. The machine had done exactly what we asked of it, 270 times.
The defect sat upstream of everything the gates could see
Province capitals and towns of under a thousand people had entered the queue at identical priority. A regional hub with real search demand and a village nobody looks up were the same kind of row to the pipeline, which had no notion of kinds of rows. It had a notion of valid rows.
That is what every gate we had built measured. Is this claim true, is this structure well-formed, is this text original, did this publish. All correctness gates — and correctness gates answer a different question from value gates. Any number of the first provides no coverage of the second. A green board proves the absence of detected error, not the presence of worth.
Nobody had estimated, before building, what a single row could be worth if the build went perfectly. Not the expected value — the best case. That number exists to identify rows whose best case is still not worth having, and those rows are invisible to a system that only asks whether output is correct.
AI content ROI is decided before generation, not measured after it
The comparison that would have saved the run could only have happened at one moment: before any of the 270 existed, with the full candidate set in front of us. A pipeline that discovers work one item at a time cannot compare items, and comparison is the entire mechanism. Once a row is built its cost is spent, and the only question left is whether it came out correct — which it did, 270 times.
It also explains why better writing would not have rescued the AI content ROI here. The guides were good; quality was never the variable. Sharper ones would have moved position 40 to perhaps 35, on rows whose ceiling was low either way.
The gate fired on its own author the first time we ran it
When we turned this into an actual pre-build gate and pointed it at our own queue, the first version passed 191 of 197 candidates. We had scored each candidate on the number of consolidated sources behind it, and the spec caps an original at eight sources. Everything with eight or more tied at the top. The score stopped discriminating exactly where the interesting candidates were.
A floor that nothing fails is not a floor. It was a budget wearing a threshold’s clothes. The repair was not a higher number but a ceiling that measures something which actually varies across candidates instead of saturating. The rerun rejected 141 of 186.
What this means if you are about to run a batch
Three things carry into any repeated build — a bulk publish, a migration, a generation run, anything where one procedure repeats across rows.
- Verify negligible unit cost against the total, not the unit. Seventy credits is nothing; 270 rows of it is 19,000.
- Record the estimate before the build even when you distrust it, because an item with no prior estimate can never afterwards be judged as anything except complete.
- Publish what you declined next to what you built. A silent cut reads later as though the whole candidate set was covered, which is the same error one level up.
Keep the honesty about your own counterfactual too. In our grounding run the roughly 200 suppressed rows were calculated, never built, and never confirmed to have failed, so every claim about the AI content ROI we saved is one unrun experiment away from being an assumption.
The runnable procedure — how to enumerate, what unit to estimate in, how to derive a floor that holds — lives in the member library.
How we know
What Google’s documentation says about measuring and scaling content
Search Console’s numbers have specific definitions, and they matter when the result is four clicks. Google’s help page explains that the position shown is the topmost position a link to your page held, averaged across all queries in which your property appeared. An average near 40 is a blend of many queries, not one ranking. The same page says an impression is generally counted when an item appears in the current page of results, so impressions are a measure of being listed, not of being seen.
Google’s guidance on helpful content is framed as a self-test. Evaluating your own content against these questions is offered as a way to judge whether a page is helpful and reliable. One of the questions asks whether the content does provide original information, reporting, research, or analysis. That is a question about value. A schema check or a fact check does not answer it.
The spam policies address volume directly. Google defines scaled content abuse as a situation where many pages are generated for the primary purpose of manipulating rankings rather than helping users. One of its examples is using generative AI or similar tools to generate many pages without adding value for users. The test is purpose and value, not the tool.
Google’s post on AI-generated content takes the same line. Automation becomes a problem when it is used with the primary purpose of manipulating ranking in search results. Generating with a model is not, on its own, what the policy is about.
How this lines up with our 270-guide run
The documentation explains why our gates could not see the problem. Every gate we had asked a correctness question: is the claim true, is the structure valid, is the text original. Google’s self-assessment and spam policies ask a value question, and none of our gates was built to ask it. A green board told us about errors. It could not tell us about worth.
The position definition explains the four clicks. At an average near 40, our own arithmetic of 0.1 to 0.2 percent click-through on roughly 4,400 impressions predicted four to nine clicks. The pages earned what the position allowed. Better writing would have moved that average a little on rows whose ceiling was low either way.
The policy’s emphasis on adding value is also why the fix sat before generation. Province capitals and villages of under a thousand people entered the queue at the same priority. No later check could have told them apart, because both kinds of row were valid. Only a comparison across the full candidate set, made before anything was built, could.
We are not claiming the guides broke a policy. Our duplication test found every pair of 354 posts below 0.10 on eight-gram overlap. What we are saying is that the documentation measures something our pipeline never estimated.
A pre-build checklist for content batches
- List every candidate row before generating any of them.
- Estimate each row’s best case in a unit that varies across candidates, and write the estimate down.
- Check that the gate actually rejects something. A floor that passes nearly everything is a budget.
- Multiply unit cost by the full row count before approving the run.
- Ask, for each row, what original information it would add that searchers cannot already find.
- Publish the list of declined rows next to the built ones.
- Ninety days after launch, read clicks, impressions and average position together, per page.
More questions
Is average position in Search Console a ranking for one keyword?
No. Google describes it as the topmost position of your link, averaged across all queries in which your property appeared. A single figure near 40 can hide a few good queries and many poor ones, so read it per query when deciding what to fix.
Does AI content ROI depend on whether Google can tell a model wrote it?
Google’s spam policies target the use of tools to generate many pages without adding value for users, whichever tool it is. In our run the guides passed every correctness gate and still returned four clicks, because the rows chosen had a low ceiling before a word was written.
Grounded in: our own content factory run — 270 guides published across twenty days, audited 14–15 August 2026, and the resulting gate run live on our skill-originals queue on 17 August 2026. Verified: ninety days of search data read back at roughly 4,400 impressions, four clicks and average position near 40; 354 posts compared pairwise on eight-gram overlap with every pair below 0.10; and 191 of 197 candidates passing the first floor before it was rebuilt and rejected 141 of 186. The runnable procedure lives in the member library.