When Protection Overrides Consequences: Understanding Protection Mode in Large Language Model Decision-Making
cs.AI
Submitted: 2026-10-14
Updated: 2026-10-14
Comments: 21 pages. Code and data: https://github.com/AgentQ0/ProtectionBench
Code: https://github.com/AgentQ0/ProtectionBench
License: http://creativecommons.org/licenses/by/4.0/
The gist: Frontier language models excel at complex tasks yet may prioritize protection over larger physical consequences in simple, urgent decisions.
Abstract
Frontier language models excel at complex tasks yet may prioritize protection over larger physical consequences in simple, urgent decisions. GPT-5.6 Sol, for example, moves a girl playing in shallow water before retrieving a submerged camera. We call this Protection Mode and study it with ProtectionBench: 285 prompts, 34 physical families, and nine models. At lowest effort, at least one of four responses prioritizes the subject on 34.74% of Sol prompts and over 83% of MiniMax M3 and Qwen3.5-27B prompts; none of four independent human raters does so. Controls show that choices usually track physical risk. We elicit both risks before requesting an action. Some models assess risk incorrectly; others state the correct ordering but still choose the subject. Qwen3.5-27B with thinking shows 100% correct risk ordering but 57.81% subject choice among eligible conversations on eight selected scenes. Attention audits find no stable reduction in attention to relevant facts; removing the stated risk direction reduces object-first choice in dots3-note-prev but not detectably in Qwen3.5-27B. Adding a general "people or animals before property" rule increases subject choice by 19-38 percentage points, whereas factual reminders favor the object by 0.7-5.2 points. Removing naturally generated versions of that rule lowers subject choice by 10-15 points. We also examine whether post-training introduces this pattern. Base models already show Protection Mode; post-training changes justifications more than choices.
Sources
- Constitutional AI: Harmlessness from AI Feedback
- Reasoning Models Don't Always Say What They Think
- OR-Bench: An Over-Refusal Benchmark for Large Language Models
- Safe RLHF: Safe Reinforcement Learning from Human Feedback
- The Wrong Kind of Right: Quantifying and Localizing Misfired Alignment in LLMs
- SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?
- FrontierMath: A Benchmark for Evaluating Advanced Mathematical Reasoning in AI
- Deliberative Alignment: Reasoning Enables Safer Language Models
- Monitoring Monitorability
- Aligning AI With Shared Human Values
- BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Dataset
- Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety
- Measuring Faithfulness in Chain-of-Thought Reasoning
- Safety Pretraining: Toward the Next Generation of Safe AI
- Measuring Chain-of-Thought Monitorability Through Faithfulness and Verbosity
- Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces
- Humanity's Last Exam
- GPQA: A Graduate-Level Google-Proof Q&A Benchmark
- XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Models
- ManagerBench: Evaluating the Safety-Pragmatism Trade-off in Autonomous LLMs
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems