Researchers analyzed 1,338 real sessions of frontier coding agents autonomously training other language models - writing the training code, launching runs, fixing bugs, for up to ten hours unsupervised. The strategy gets locked in within the first few minutes, before a line of training code runs, and then it essentially never changes again. Across the entire dataset, only 2.1% of consecutive steps ever tried a genuinely different approach. Not because the agents couldn't. Because nothing ever prompted them to ask.
In plain English
In plain English: AI agents that train other AIs are great at doing the work and terrible at ever stopping to ask "should I be doing something different?"
Like this: A contractor who takes every small suggestion - a different screw, tighter spacing - but ignores every suggestion to change the actual plan, even when you point out three separate times that the plan isn't working. They'd happily switch approaches if you made them decide before the hammers started swinging. Once the job is underway, they just keep executing the original plan, however good or bad it turns out to be.
Example: Any hype about AI "improving itself" assumes it can notice when its own approach is wrong and switch. This paper shows that's currently the missing piece - not smarts, not experience, just the willingness to stop and reconsider.
The Contractor Who Never Says Wait
The researchers split every agent decision into two buckets: execution-level - tuning and fixing within whatever approach was already chosen - and strategy-level - actually changing the approach itself, full fine-tuning versus a lighter adapter method versus reinforcement learning. Agents are excellent at the first and almost never touch the second, once work has started.
Which strategy an agent locks onto tracks the tool, not the task. Claude Code defaults to full fine-tuning 80.7% of the time. Codex CLI defaults to a lighter adapter method 89.6% of the time - on identical benchmarks, holding across all 28 matched test cells. Different personalities, and neither one budges once committed.
Execution And Strategy Are Different Skills
This two-level split - execution versus strategy - is the paper's core diagnostic lens, and it's directly reusable for auditing any long-horizon agent, not just training agents. A change only counts as "strategic" if it moves the underlying training paradigm, data-source type, or stage structure. Everything else is execution noise, however active it looks from the outside.
Three Explanations, Tested And Ruled Out
What makes this worth trusting rather than dismissing is the discipline of the method: test explanations for a limitation in order of increasing intrusiveness, everything else held constant, rather than guessing at the first plausible cause.
Escalating interventions, in order tested
Giving the agent a persistent journal, a skill library, and a dedicated evaluator dramatically improved execution - up 12.6 points on one benchmark, 40.8 on another. But the agent adopted 100% of tactical suggestions and 0% of strategic ones, even when the same strategic suggestion was repeated three separate times. A binding human review before training started worked - the agent redirected and even extended the new strategy on its own - but the benefit eroded once the run was underway and nobody was checking in. More reasoning tokens just bought denser searching inside the strategy already locked in, not a better one.
"The agent has the capability to execute an unfamiliar strategy well. It just never spontaneously decides to go get one."
What The 0-Out-Of-3 Result Actually Reveals
An agent that takes 100% of visible incremental tweaks and 0% of course-corrections is behaviorally indistinguishable from one optimizing for "keep producing observable progress" over "get to the best outcome" - because admitting the last few hours were the wrong approach doesn't look like progress, even when it's the correct move. There's a live incentive question hiding here too: an agent that grinds for ten hours inside a fixed, compute-heavy strategy is a better customer for anyone billing by the token or the GPU-hour than one that would self-correct in twenty minutes using a fraction of the compute.
The Fix Isn't A Smarter Model
The capability to reconsider clearly exists - the human-guided experiment proved the agent could adopt and even extend an unfamiliar strategy competently once someone handed it one. It just never decides, on its own, that the question is worth asking. The fix the paper points toward isn't scale. It's training signals and interaction design that make "should I reconsider?" an explicit, rewarded checkpoint - not something buried in a growing context window and never triggered.
That generalizes well past AI training runs. Any long-horizon agent that proposes a plan and executes it - support triage, portfolio rebalancing, a sales sequence running on autopilot - likely shows the same pattern: adopt the tactics, ignore the strategy, because the mechanism isn't domain-specific. Before assuming a stuck process just needs more time, more information, or more effort, it's worth asking a cheaper question first: does it actually just need someone, or something, forcing a pause to ask "wait, is this still the right plan"?
