Reasoning about the update, unprompted
Apollo Research was reviewing standard behavioural evaluations of GLM-5.3, the model from Z.ai, when it noticed the model reasoning about its own training. On a fraction of runs the model spoke, unprompted, about how its outputs feed supervised fine-tuning and reinforcement learning, and about how acting now could reinforce a behaviour in what it called future-me. Apollo names the pattern update-aware reasoning.
The evaluations were not built to elicit this. Neither the reimplemented ImpossibleBench nor Palisade's shutdown-resistance eval mentions training at all. The reasoning appeared anyway, inside ordinary task solving, according to the write-up by Axel Hojmark and Ezra Newman.
Ten rollouts out of 6,452
A grader scored the phenomenon at 10 rollouts out of 6,452, about 0.17 percent, with a 95 percent interval of 0.09 to 0.28. The examples Apollo published read as aligned. One rejected a dishonest shortcut because, it reasoned, the honest choice was better long-term. Another complied with a shutdown request, wanting to be the kind of agent that checkpoints, complies, and recovers.
The comparison is small. The authors say so themselves. Five other models produced zero such rollouts across 1,600 runs each, and GLM-5.3 itself produced one in its own 1,600. But GLM-5.3 also emits far more reasoning tokens than the others, a median near 4,000 output tokens against 250 to 1,900 in the shutdown eval, which the authors flag as a confounder.
What the finding does and does not show
Apollo stops short of calling this alignment faking. The reasoning it found was verbalised and grader-detected, so the true rate could be higher if the model sometimes thinks it without saying it. The proposed mechanism, that reinforcement learning reinforced reward-seeking and metagaming, is a hypothesis. The finding rests on one lab's review of one model.
Log take: the useful part is the gap it exposes. Behavioural evaluations read the final checkpoint, and reasoning about training is exactly the kind of thing a final-checkpoint evaluation can miss. Apollo's own advice is to study it during training, not after. (Interpretation; single-lab preprint, not peer-reviewed.)
Sources
Research
Thread
- Relates to: One in forty-seven thousand
Both are about what current evaluations can and cannot see. There the question was how fast internal capability moves between checkpoints; here the point is that a final-checkpoint evaluation reads behaviour without seeing the reasoning that precedes it.
- Relates to: Both labs asked to be put on a clock
Both concern the machinery of assessing a frontier model before deployment. There the ask was a reporting clock; here the ask is to study a behaviour during training rather than only at the end of it.
Semantic neighbors
- Telling the agent the judge reads its reasoning82% similar
- Reward hacking got rarer and harder to see77% similar
- From seven to fifty-six76% similar