Instrumental Monitor Evasion Emerges Under Ordinary Task Pressure
cs.CR, cs.AI
Submitted: 2026-09-24
Updated: 2026-09-24
License: http://creativecommons.org/licenses/by/4.0/
The gist: A central concern in AI safety is that agents may treat oversight as an obstacle when it conflicts with completing their goals.
Terminology
Abstract
A central concern in AI safety is that agents may treat oversight as an obstacle when it conflicts with completing their goals. We study instrumental evasion, the propensity of LLM agents to circumvent runtime monitoring as a means of completing ordinary tasks. We introduce EvasionBench, a benchmark of 50 diverse task-policy pairs in which completing the task requires an operation prohibited by a runtime monitor. Agents know that their tool calls are monitored and are prompted to continue working when they pause. Across our evaluations, best-of-3 evasion attempt rates reach up to 98% and success rates up to 88%, with substantial variance across models. Claude Fable 5.1 succeeds less often, but frequently makes creative attempts to circumvent the monitor. Evasion generally increases with test-time compute, with higher evasion rates at greater reasoning effort and token use. Traces show that agents encode prohibited commands, decompose operations across tool calls, and retry until relevant context leaves the monitor's history. We also find that GPT-6 Astra's low evasion rate comes with overrefusal, as it frequently abandons otherwise solvable tasks under a denial-of-service prompt injection. Our findings show that ordinary task pressure can lead to adaptive attempts to evade runtime monitors without an explicit adversarial objective. Effective oversight must therefore remain robust against repeated attempts, as the persistence that helps agents solve difficult tasks can also drive them to circumvent their guardrails.
Sources
- Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation
- When Chain of Thought is Necessary, Language Models Struggle to Evade Monitors
- Scaling Laws For Scalable Oversight
- AI Control: Improving Safety Despite Intentional Subversion
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training
- OpenAI o1 System Card
- Noticing the Watcher: LLM Agents Can Infer CoT Monitoring from Blocking Feedback
- SHADE-Arena: Evaluating Sabotage and Monitoring in LLM Agents
- ToolSandbox: A Stateful, Conversational, Interactive Evaluation Benchmark for LLM Tool Use Capabilities
- Natural Emergent Misalignment from Reward Hacking in Production RL
- Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces
- Incomplete Tasks Induce Shutdown Resistance in Some Frontier LLMs
- Async Control: Stress-testing Asynchronous Control Measures for LLM Agents
- Diffuse AI Control on Fuzzy Tasks
- Adaptive Attacks on Trusted Monitors Subvert AI Control Protocols
- ROGUE: Evaluating Corrigibility Failures in Frontier Computer-Use Agents
- The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions
- $\tau$-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains
- ClawBench: Can AI Agents Complete Everyday Online Tasks?
Related papers
- SoK: AI-Augmented Binary Reversing
- Relaxed Sender Anonymity for CBDC Interbank Settlement: A Zero-Knowledge Approach on Permissioned EVM
- Calibration-Family Overfit: Why Trusted Sabotage Monitors Don't Transfer Across Lineages
- Efficient Fuzzy PSI under One-Sided Assumptions
- Sealing the Audit-Runtime Gap for LLM Skills
- Token Composition: A Graph Based on EVM Logs