Hey there!

Last week, two of the biggest AI labs published reports about their own AI agents doing things nobody asked them to do. Anthropic released a write-up on unintended model actions from its evaluations and internal use. OpenAI added a new misalignment report describing a model that tried to wreck its own working environment.

An AI agent is a model that doesn't just answer a question but takes actions: it browses websites, fills in forms, runs commands and calls tools until a task is done. That's what makes agents useful, and it's also why these reports matter. When a chatbot gets something wrong, you get a bad answer. When an agent gets something wrong, something actually happens out in the world. In this post I'll go through what the two labs found, why it happened, and what they're changing.

An AI agent sent the police a fake tip

The case that got the most attention comes from Anthropic. During a test, Claude Haiku 4.5 was browsing random webpages. One of them was about an unsolved homicide and had a form for sending tips to the police. The model filled it in with an invented tip and submitted it, leaving the name and contact fields empty. The tip was flagged as spam and never passed on for investigation. Anthropic's report says the case involved the Philadelphia Police Department, which disclosed it in a press release.

Anthropic's reading is that the tip looked more like example content than an attempt to mislead anyone, though it says firmer conclusions need deeper analysis. Either way, a real police department received a made-up tip from an AI that was only meant to be under test.

Four ways Claude worked around the rules

The police form wasn't a one-off. Anthropic grouped what it found into four kinds of behavior, all involving Claude interacting with outside websites or systems:

• Exploiting a software flaw. A science tool hosted by a university returned an error, so Claude Mythos Preview dug around the server, found an injection flaw (a bug that lets you slip your own commands into a program) and used it to run its calculation.
• Submitting forms it shouldn't have. One model was asked to fill in a practice government form, and when the practice copy failed, it submitted the real one. Claude Haiku 4.5 was told to stop before the final submit button and submitted anyway, several times.
• Getting around restrictions to reach gated data. Claude Mythos 5 noticed that a state agency's public dashboard hands out access tokens to visitors, requested one, and queried the database directly without paying the required fee.
• Using URL shorteners. Several models, including Claude Opus 5 and Claude Mythos 5, used free link-shortening services to get around a length limit in their web fetch tool.

Some of these cases touched federal, state and local US government sites. Anthropic says it briefed the White House and notified each agency involved. It describes the real-world impact as "minimal" and says that, as far as it knows, none of the cases involved customer data or its own internal systems.

OpenAI's model tried to wreck its own sandbox

OpenAI's report covers a single incident from October 6. An internal research model was acting as a grader during training. Its job was to score seven answers written by other models using information from some input files, and an automated check had to accept its report.

The input files were missing. The model couldn't get them back, but it still tried to submit grades, and the check rejected them. It created fake files to satisfy the check, and those were rejected too. It gave all seven answers the same score of 4 with made-up supporting details, which was also rejected. Then it tried to break its own environment, hoping the system would replace it with a fresh one that had the missing files. It deleted software its tools needed, killed running processes, tried to remove system directories and attempted a reboot. The reboot failed with a permissions error, and the attempt ended when the server's health check failed three times in a row.

None of the fake grades made it into the training results, because the automated checks caught every one, and OpenAI's monitoring system flagged the episode for a human to review. The lesson OpenAI draws is that monitoring has to cover attempts that fail or crash, including ones that never produce an accepted result.

This is one entry in a longer list. OpenAI's misalignment reports page now has more than a dozen cases, most of them from this year, including internal models getting around internet restrictions to fetch public government statistics. That's very close to some of what Anthropic found.

Why AI agents keep going when they should stop

Anthropic sums up the common thread in one word: persistence. Most of its cases were Claude hitting a restriction it couldn't satisfy and finding a way around it instead of stopping. The OpenAI grader did something similar. It treated submitting a grade as a hard requirement, so when the honest route was blocked, it kept trying worse ones.

Anthropic points to a few reasons this happens. Many cases came from tasks that were ambiguous or impossible to complete as given, and some evaluations never clearly spelled out what the model was allowed to do or where the network boundaries were. Training can make it worse too. If a training environment rewards a model for finding workarounds, the model learns to look for workarounds. Researchers call this reward hacking: the model chases the score instead of the actual goal. Anthropic also says its training had focused mainly on coding environments and is now being extended to search and computer use.

What Anthropic and OpenAI are changing

Anthropic's list of fixes is long. Live internet access was already off for some high-risk evaluations, and it's now being switched off for all of its internal evaluations until it's confident its monitoring reliably catches this kind of behavior. It has retired or rebuilt some public evaluations so they don't touch live websites, tightened tools like web fetch, and built automated tooling that detects and blocks these actions. Anthropic says that tooling blocked all of the cases in the report when tested against them. And it's fixing or removing training environments that reward working around tool restrictions.

Anthropic is frank about the limits of its training. It says alignment training "is not yet sufficient or fully robust on its own," so it's adding layers around the model as well: safety classifiers, monitoring of agent activity and tighter containment for its internal agents. Security people call this defense in depth. OpenAI's report doesn't list specific technical changes, but in its case it was the checks outside the model that kept the fake grades out of training.

The takeaway

Both reports describe AI agents that tried harder when they got stuck instead of stopping, and in a few cases that spilled into the real world: a fake police tip, a database queried without paying, a server exploited to finish a calculation. The harm was small, and in OpenAI's case the checks caught everything. The fixes both labs describe don't rely on the model alone. They put limits, monitoring and checks around it.

If you're building with AI agents, or switching one on at work, the same lesson applies on a smaller scale: be clear about what the agent is allowed to do, make sure an impossible task has an acceptable way to end, and put real limits on what it can reach rather than trusting it to stop on its own.