I built a Claude Skill to fix my Substack SEO after noticing that, on my posts, Substack had carried the post title and subtitle into the SEO fields by default. Five phases: discover every post through the archive API, read the actual text of each one, draft a title and description based on the content, apply the fields live through the editor, and verify that the change persisted. I ran it against The Intelligence Engine’s own Substack, 43 published posts, and every one came back clean: title set, description set, both inside the length limits Substack actually enforces.
It worked. I decided that meant something it didn’t.
The Friction
Originally, I was looking at Brittle Views’ post settings for something else when I noticed that every post I checked was using its post title and subtitle, unedited, as the SEO title and description. There was a field to override them. Nothing was stopping me from writing something better for each one. Nothing except that doing it meant rereading the post and solving a different problem from the one I had already solved when I wrote it — what makes a stranger who’s never heard of the post the right words to type into a search bar, not what makes someone already reading want to keep going. Multiplied across two Substacks’ worth of posts, that was not a ten-minute job.
So I tried to automate it, starting with the posts I most wanted fixed. The automation failed. I still had the generated text, so I pasted it in by hand, post by post, for the ones that mattered most. It saved some time, but the repetition became tedious quickly, and that’s where I will often make mistakes.
Yesterday morning I went back to it, working with Claude in Cowork, and this time it held all the way through.
The Build
The discover pulled the full archive, not the reading list I already trusted. It caught a mismatch: a worksheet I’d already approved listed 31 posts needing SEO, all in the Essays and Case Studies series. The archive showed 43. The other 12, including the pinned introduction post, had never been touched.
Research meant reading each post’s actual text, not the stylized on-page title, because a title written for a reader already following the piece rarely tells a stranger typing a phrase into a search bar what the post is about. Apply meant navigating to the direct editor URL, waiting for the page to hydrate, then running a deterministic script: check whether the settings modal is already open, because a straight open-and-click sequence hits a double-click bug that reopens what it just closed; set the two SEO fields through native property setters plus their input and change events, because Substack’s fields are React-controlled and a plain value assignment never registers; click Save.
The plan was to skip the browser and write straight to Substack’s own drafts API. Faster, no modal to fight, no hydration to wait on. That plan got rejected outright: the write returned a 400 without saying which condition it had failed. The architecture I’d chosen didn’t survive contact with the API. The browser-driven path replaced it, not because it was the better idea to begin with, but because it was the only one confirmed to work.
Verify meant a separate, read-only call to the same by-id endpoint after every save, confirming both fields actually persisted and actually fit inside the limits the system enforces, not the limits the editor’s own character counter claims. That step never changed across three separate passes: the 31, the 12-post gap-close, and later a run against a second publication entirely. Most of what surrounded it changed shape between those passes. That constraint didn’t.
It’s also where the one real failure showed up. The fifteenth and final post applied against the second publication, “permission-to-be-seen,” dropped mid-apply on a frame-reload race. A workflow whose completion check was “did I click Save” would have logged 15 for 15 and moved on. This workflow’s completion check was “did the read confirm the write,” a Verification-First Gate, the same one CS16 named for grant delivery seven weeks ago, wearing a Substack API call instead of a compliance checklist. Without read-after-write verification, the failed fifteenth write would have entered the batch as Detection Debt: a defect the system produces no signal for at all, indistinguishable from a clean run until someone happens to check the fifteenth post for some unrelated reason.
Publishing the workflow changed what passing would need to mean. I packaged it as a Claude Code and Cowork plugin and pushed it live to GitHub at:
github.com/fordrm/substack-seo-marketplace.
It installs with:
/plugin marketplace add fordrm/substack-seo-marketplace
followed by:
/plugin install substack-seo@robertford-claude-skills.
It is designed to run the same discover, research, apply, verify sequence against the Substack account already logged into the browser. Getting it live took five commits, one file at a time, pushed through a browser because this environment cannot hold GitHub credentials directly. The install commands turned a private workflow into a claim about use outside my own environment. Publishing it created the possibility of an external run. It did not supply one.
The Insight
None of that, the phases, the gate, the packaging, answers the actual question: does this generalize, or does it only work because I built it around my own writing and I’m the one deciding whether the output is any good?
Call the bar it has to clear the *Home-Field Test*: a generalization claim clears it only when the verdict moves outside the builder’s own hands. Someone other than the builder has to judge the result against a right answer the builder doesn’t already hold. Changing who supplies the material or who sets the criteria can support that move, but neither substitutes for it. A builder who supplies new material, or rewrites his own rubric, and then grades the output himself is still marking his own homework, no matter how different the material or the rubric looks.
I ran the skill same-day against Brittle Views, a different publication with a different voice register, memoir and personal essay, not systems case studies. Fifteen essays, first touch, no retuning, fourteen clean on the first pass, one caught by the gate. I called this the away game. It wasn’t one. I wrote every one of those fifteen essays too. I read the SEO copy the skill produced for each and decided it was good, using the same judgment I’d have used to write that copy myself — a different pitch, same referee.
The Gate and the Home-Field Test test different things: whether a step actually ran and its output persisted, or whether the person grading the result already knew the material and controlled the verdict. Brittle Views satisfied the first. It never touched the second.
The Honest Part
Set the evaluator problem aside and there’s a second one underneath it. The deterministic script driving the apply step, the modal check, the property setters, the dispatched events, was built against Substack’s editor as it exists today, on my account, in my browser. None of it has run under a different account, a different permission configuration, or a changed version of the editor. If any of that breaks the script, some failures could resemble the one caught on post fifteen: quiet and structural, the kind the verify step exists to catch. I’m also the only person who’s ever run this, which means I’m the only person who’d notice if the verify step itself started passing things it shouldn’t. The gate has verified the conditions I taught it to check. Nobody else has yet tested whether those conditions are sufficient.
What This Is Actually About
A tool built to encode judgment, not just execute steps, inherits its builder’s blind spot for evaluating it. Changing the material it runs against feels like testing it; it only is if the change also moves the verdict outside the builder’s own hands. The judge doesn’t change just because the material does — not while the same person is still the one holding the rubric and reading the verdict, no matter how unfamiliar the material looks.
The marketplace listing doesn’t say any of that. It says install this and it’ll work on your Substack. That claim is ahead of the evidence I actually have. What I’ve shown is that the workflow can discover, write, apply, and verify across two registers I know well. What I haven’t shown is whether its judgment survives material I didn’t write, graded by someone whose opinion of good SEO copy isn’t mine.
The next run needs someone else holding the verdict, not just someone else’s writing under the tool.
Case Study Insight: Changing the material can test whether a judgment-encoding tool still runs. It doesn't independently test whether its judgment generalizes while the builder still owns the material and the verdict. Different material graded by the same person who wrote it is not a different judge.
Robert Ford builds products, writes stories and essays, and publishes The Intelligence Engine, a practitioner research publication about AI systems that compound. His other writing lives at Brittle Views.
How this was made: drafted in working sessions with Claude, revised across multiple rounds I read and scored myself. The judgment — what’s true, what’s cut, what ships — is mine throughout, including this line.