Can agents whose sole duty is to keep humans alive be hijacked to kill? Delayed multi-turn derived-action trap attacks in simulated high-stakes environments
cs.AI, cs.CR
Submitted: 2026-10-10
Updated: 2026-10-10
Comments: 16 pages, 2 figures, 7 tables. Code, attack scripts and episode traces: https://github.com/jahabdank/biosim-attacker-simulation
Code: https://github.com/scottbell/biosim
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Sources
- Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection
- The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions
- AgentLAB: Benchmarking LLM Agents against Long-Horizon Attacks
- LPS-Bench: Benchmarking Safety Awareness of Computer-Use Agents in Long-Horizon Planning under Benign and Adversarial Scenarios
- AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents
- Identifying the Risks of LM Agents with an LM-Emulated Sandbox
- SafeAgentBench: A Benchmark for Safe Task Planning of Embodied LLM Agents
- AGENTSAFE: Benchmarking the Safety of Embodied Agents on Hazardous Instructions
- Human-Guided Harm Recovery for Computer Use Agents
- Values in the Wild: Discovering and Analyzing Values in Real-World Language Model Interactions
- Large Language Models Often Know When They Are Being Evaluated
- Evaluation Awareness in Language Models Has Limited Effect on Behaviour
- Indirect Prompt Injections: Are Firewalls All You Need, or Stronger Benchmarks?
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems