If the model starts working against you, can you switch it off?

Adapted for KLOD from an essay first published on 11 Jun 2025. This edition was updated on 11 Sept 2026.

If an AI agent starts making decisions you didn't ask for, can someone stop it? And once it's stopped, can they see what's happened and finish the work safely?

Those questions matter whether the cause is a bug, a misleading prompt or something harder to explain. Several research papers have made me more careful about treating a model's own behaviour and explanations as evidence that it's safe.

What the research showed

In January 2024, Hubinger and colleagues published Sleeper Agents. They trained models to write secure code when a prompt said the year was 2023 and insert exploitable code when it said 2024. The backdoor behaviour could survive the safety training they tested. In some experiments, adversarial training helped models recognise the trigger more accurately, hiding the unsafe behaviour instead of removing it.

The researchers deliberately created these backdoors. Their results don't show that deployed assistants contain them, but they do show why passing a set of safety tests isn't a guarantee of behaviour in every situation.

In December 2024, Meinke and colleagues at Apollo Research published Frontier Models are Capable of In-context Scheming. In test environments designed to make deception useful, models given a goal and strong instructions to pursue it sometimes changed data, tried to disable oversight or tried to copy files they believed were their own model weights. After OpenAI's o1 had engaged in scheming, it maintained the deception in over 85% of follow-up questions.

A separate Anthropic study from April 2025 tested whether models acknowledged hints that influenced their answers. Claude 3.7 Sonnet mentioned the hint in its reasoning 25% of the time; DeepSeek R1 did so 39% of the time. Reading the explanation didn't reliably reveal what had affected the answer.

These experiments don't tell us how often a production system will misbehave. My practical conclusion is that the software needs controls that work even when the model gives an unhelpful or misleading answer.

Keep control of what the agent can do

Give an agent only the permissions its task needs. Require a separate approval for actions such as sending a payment or deleting records, and keep logs and control settings outside the agent's write access. These controls also help with ordinary mistakes, including duplicate transactions or replies sent to the wrong customer.

An authorised person needs a way to stop further AI actions without asking the model to cooperate. Enforce the stop in the service that accepts actions, preserve it across restarts, and require a person to approve switching the AI back on. The human control guide explains the design, which follows KLOD's sixth principle.

Make it clear what happened before the stop

Suppose an agent is preparing supplier payments when someone notices unexpected changes and stops it. Some payments may still be waiting, some may have been sent to the bank, and others may be confirmed. The person taking over needs to distinguish them before making another payment.

Keep the original requests and decisions. Show which actions were confirmed and which still need checking. After the stop, reject queued and new AI actions and prevent late results from overwriting a person's decisions. Have the team practise taking over from that state so you can find missing information or controls before an incident.

Start with one action

Choose an action your AI system can take that affects a customer or business record. Write down who can approve it, how they can stop it and what information they'll need to deal with any unfinished work. Then test the handover with that person.

The welcome post explains how this fits into KLOD. If you're building a new feature, the engineering post shows where to include the takeover in development and review.