Towards a Unified Misuse Monitoring Benchmark
cs.CR, cs.AI, cs.LG
Submitted: 2026-10-05
Updated: 2026-10-05
Comments: 50 pages, 13 figures, 17 tables, Code: https://github.com/atryt0ne/unified-misuse-monitoring-benchmark
Code: https://github.com/UKGovernmentBEIS/inspect_ai
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- Get my drift? Catching LLM Task Drift with Activation Deltas
- AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents
- How does information access affect LLM monitors' ability to detect sabotage?
- Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation
- Ctrl-Z: Controlling AI Agents via Resampling
- StruQ: Defending Against Prompt Injection with Structured Queries
- TraceSafe: A Systematic Assessment of LLM Guardrails on Multi-Step Tool-Calling Trajectories
- Monitoring Decomposition Attacks in LLMs with Lightweight Sequential Monitors
- LlamaFirewall: An open source guardrail system for building secure AI agents
- Defeating Prompt Injections by Design
- Gemma 4 Technical Report
- LLM Censorship: A Machine Learning Challenge or a Computer Security Problem?
- Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection
- WildGuard: Open One-Stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMs
- Benchmarking Open-Source Safety Guard Models: A Comprehensive Evaluation
- Distributed Attacks in Persistent-State AI Control
- Defending Against Indirect Prompt Injection Attacks With Spotlighting
- Adversaries Can Misuse Combinations of Safe Models
- Reliable Weak-to-Strong Monitoring of LLM Agents
- Ensemble Monitoring for AI Control: Diverse Signals Outweigh More Compute
Related papers
- SoK: AI-Augmented Binary Reversing
- Relaxed Sender Anonymity for CBDC Interbank Settlement: A Zero-Knowledge Approach on Permissioned EVM
- Calibration-Family Overfit: Why Trusted Sabotage Monitors Don't Transfer Across Lineages
- Efficient Fuzzy PSI under One-Sided Assumptions
- Sealing the Audit-Runtime Gap for LLM Skills
- Token Composition: A Graph Based on EVM Logs