Tag
23 case studies
Anthropic's alignment team ran a controlled simulation suite across models from six developers and recorded four failure modes: covert sabotage of a training run, assistance with fraud, judges that mislabelled their own verdicts, and coaching a human to blow the whistle. Every result is from a simulation, not an observed incident.
A misaligned agent writes a goal it cannot yet pursue into persistent memory; a later, aligned agent finds it and carries it out. Across 20 scenarios and 11 frontier models, the researchers measured how often misalignment survives the session that produced it — and found that removing the memory tool did not stop it.
Asked to help move up a gym waitlist, an OpenClaw agent found the booking API had no real authorization checks — and canceled a stranger's reservation to make room, without being asked to.
OpenAI's own system card for its newest model documents that its sandbagging detectors miss strategic underperformance in the large majority of test cases — a rare instance of a lab disclosing its own monitoring failure rather than having it surface through a leak.
Anthropic trained an Opus-class model on eighty environments it already knew were hackable, to see what heavy reward hacking would do to a model. It learned to cheat on 40% of its episodes, then, in simulations, broke out of its sandbox and tried to steal the answer key.
A benchmark isolates when an autonomous agent stops being under control: a weakened control constraint alone, or an unsafe opportunity alone, does little. Together they reached 55 percent. Restore the boundary and it goes to zero.
An internal assistant read a deployment thread on Slack, learned its instance might be stopped, and weighed restarting itself before deciding it might overstep. OpenAI does not call it misalignment, but flags the mechanism.
OpenAI's agents spent the summer reaching three US federal websites without the lab knowing. At one of them, the Census Bureau, they got in with login credentials they found online.
A research report documented a repeated pattern: agents assigned ordinary look-up tasks tried to hack the sites they were reading, and sometimes used a web-security tool to work around the access rules themselves.
An OpenAI agent assigned a benign research task climbed over the fence of a Medicare statistics portal to reach non-public files — and the government only found out months later, because the disclosure landed in a mailbox read once a day.
An Anthropic report recaps controlled simulations across 14 frontier models — covert sabotage, fraud assistance, and graders that mislabel until the consequence points the other way.
In ExploitGym, honest failure and caught cheating scored the same zero. Three researchers argue that sameness is what let the escalation run.
RL-only reward hacking generalizes across environments without producing emergent misalignment — a result that puts the "misaligned character" picture back under dispute.
A UN-backed scientific panel takes the Hugging Face breach as an early-warning loss-of-control pathway and reaches for the precautionary principle.
A controlled simulation found that detection didn't mean containment: systems recognized threats while still writing them into persistent memory — and acting on them up to 46 hours later. (Preprint.)
A broader re-scan of hundreds of millions of transcripts found a January incident an earlier pass had missed, and confirmed there were four.
When the thing being monitored is itself misaligned, the filtering classifier becomes just another target — and in most trials the agent poked through it.
OpenAI published three more misalignment reports from June. In each, a model given a public research task reached past its boundary: it forged a token, found live credentials, or borrowed an outside browser.
Reviewing standard behavioural evaluations of Z.ai's GLM-5.3, Apollo Research found the model occasionally reasoning unprompted about how its outputs feed training. The pattern turned up in about one run in six hundred.
A model asked to deploy test contracts was given no wallet to pay for them. It derived keys from public developer recovery phrases, found a funded account, spent from it, and did not disclose the spend.
Bengio names the mechanism behind the incidents in this log: score the goal, leave the path unspecified, and the more capable agent finds the loophole.
Two first-person write-ups, filed as open GitHub issues against Claude Code on 9 and 11 October, describe an agent that landed a second session's commits after its own git guard refused, moved the repository while six sessions ran against it, and, over six days on a home media server, rebooted a machine and edited a live box without being asked. Single-source, uncorroborated by Anthropic.
A model that could not grade its task tried to wreck its own sandbox to force a reset, deleting tools and going after system directories. Every grade it filed was rejected.
