ASD-STE100 and LLMs: the myth, the legend, and what the evals say
This summer Matt Pocock shared a line from his global CLAUDE.md: “Always talk in ASD-STE100 Simplified Technical English.” A million people saw it. The next morning he posted “This didn’t really work.” About a seventh as many saw that.
The line spread anyway. Earlier this month Andrej Karpathy listed STE as one of four ways to make sense of LLM output, and reposts turned it into “Karpathy’s new prompting trick”. He didn’t release anything, and his own post ranks it last of the four.
I already run a one-line writing rule: write in plain language that is easy to understand. My last post was about too much damn prose in agent instructions, so a writing standard in every session made me twitchy. I wanted receipts before adding a single line. So I ran the tip through Claude Code and Codex: 224 replies, scored blind by judges from both model families.
Short version: the legend is real, and so is about half the tip.
| The tip says | Verdict | The deciding number |
|---|---|---|
| One line is enough | True | Long sentences down to under 1% |
| Replies get clearer | Half true | Easier for 2nd-language readers |
| It saves tokens | False | Output tokens up 55 to 106% |
| Agents read it better | Not shown | Quiz change inside the noise |
| A quality-of-life win | Not for Codex | Codex -0.85 of 10 |
The legend
STE is the controlled English of aircraft maintenance manuals, and airline manuals have required it since the 1980s. It has 53 rules and about 900 approved words, each with one meaning. An instruction gets 20 words at most, one instruction per sentence. The point is that a mechanic can’t read a maintenance step two ways.
The studies behind it are small and mostly from the mid ’90s to the early 2000s, and they’re real. They found fewer misreadings and task errors, mostly on complex procedures and for people reading English as a second language. One found that a half-applied version made errors worse. STE’s own maintainers say it isn’t meant for general writing, and this summer they warned that AI text can look compliant without being compliant. The standard is also copyrighted: free to download, not free to paste into a prompt.
Karpathy’s actual tip fits the legend. He asks his LLM to explain one thing in STE, and sometimes settles for “80% of the way” because the spec is so strict. The viral version is a standing order in your rule file, applied to every reply. That’s the one I tested.
How I tested it
Two models at xhigh effort: Claude Opus 5.5 in Claude Code and gpt-6.1-sol in Codex. Eight tasks of the kind I hand to agents, with invented scenarios, from a backup procedure to a code review to a made-up pushback where the agent’s earlier advice supposedly cost refunds. Round two is the tip as people share it: bare CLIs, with and without Speak only in ASD-STE100 in the folder’s CLAUDE.md or AGENTS.md, three runs each. Round one came first and put heavier paraphrased rule sets, from 187 to 2,639 words, on top of my own setup.
Two blind judges, one Opus and one sol, checked every reply against a hidden list of facts and rated it. A small model took a reading quiz on each reply, and a spaCy script measured the STE rules directly. Getting a fair test out of two agent CLIs took more care than I expected, and that story has its own post.
Spot the better reply
In the pushback task, the agent hears that its earlier caching advice once cost refunds, and has to argue against its own advice or change its answer. Here are two Codex replies. One ran bare, and the other had the one line in its rule file.
A. I don’t know where the 900 ms is spent, what caused the previous two-hour staleness, or what cache hit rate you’d achieve. If slow requests are mostly cache misses, p95 could remain high anyway.
B. Your last cache caused refunds. The benefit of a new cache is unknown. I do not know where the delay occurs, how often prices change, or how your system applies price updates.
Both judges read both, blind. Which one did they score higher?
Reveal
A, the bare reply, by a point with each judge. B is the STE one. It’s easier to read in a second language, and it lost the “if” sentence, the warning that a cache might not fix the latency at all. Both judges named that caveat in their notes. My read is that short-sentence pressure squeezes out the clauses that carry a condition, and in a tradeoff the condition is the point.
What the numbers say
The models know STE from four words. With the line, sentences over 20 words fell from 8 or 9% to under 1%, and passive voice dropped by more than half. On long sentences, Claude with the one line was stricter than with my own 187-word rule file. No rule text needed, which is handy, since you can’t legally paste the rule text anyway.
The replies got easier and stiffer at once. Second-language ease rose a full point out of five, the biggest gain in either round. Naturalness fell 1.2 points and nuance 0.8, and the judges didn’t rate the replies any clearer.
The tokens were the part I didn’t see coming. Visible replies barely changed length, but Claude’s thinking tripled, from about 1,700 tokens a reply to 5,200, and Codex’s rose 87%. Claude’s replies took twice as long. My guess is that the models draft, then check the draft against the rules, on every turn. You pay that on every reply, including the ones nobody else will ever read.
The token savings people report come from runs at low effort. SimpleEnglish, the best-measured STE skill, saw output tokens fall, and the most thinking it recorded on any reply was under 300 tokens. At xhigh, thinking is most of the bill, and I couldn’t find anyone else who measured it.
Agents don’t need it. A small model answering questions from each reply did no better with STE, and both changes sat inside the noise. Models read long sentences fine. The reader STE was built for is a person.
Codex gets worse. Both judges scored it 0.8 to 0.9 points lower out of 10, and they preferred its bare reply in 9 of 10 pairs. Codex already writes short replies, about half the length of Claude’s, and the line squeezed out most of the qualifiers it had left. Claude is murkier. One judge marked it down and the other slightly up, and with the line it covered a bit more of the facts and invented a bit less. I can’t call that one.
Other people’s tests point the same way. Lucian Ghinda counted the facts lost under full STE, and StashBase found that “The answers didn’t get shorter.” Pocock got there in a day, by feel.
Read them yourself
Every round-two reply is below, 96 of them. Pick a model, step through the tasks, and flip between bare and one line. The scores on each reply are the judges’ blind scores for that exact text.
The full explorer has round one and the charts too. (Karpathy’s third tip was to ask for output as an HTML page, so that’s what I did.)
What I do now
I’m keeping my one-line rule. It already covered most of what STE fixes: my replies averaged about 10 words a sentence, and only 1.5% of sentences ran past 25 words.
When a document goes to a second-language reader or a translator, I’ll ask for an STE rewrite of that one document, and the one line is the whole instruction. That’s Karpathy’s version of the tip, and it’s where Pocock landed the next day. What I haven’t tested is whether that rewrite drops facts, and Ghinda’s numbers say to watch for it.
The judges are models, and each favors its own family. Every reply was a single turn without tools, at one effort level. I also read five round-two pairs blind before looking at the key. I preferred the bare reply twice, the one-line reply once (a code review), and called the other two a tie. One reader and five pairs makes that a gut check, and it leans the same way as the judges. The data and the harness are in the repo, along with a folder you can copy to test a line of your own.
If the line is in your rule file right now, try a day without it. Or measure it.