SoK: When Safe Agents Fail Together: The Security of Multi Agent LLM Systems
cs.CR, cs.AI
Submitted: 2026-09-01
Updated: 2026-09-01
Comments: 21 pages
License: http://creativecommons.org/licenses/by/4.0/
The gist: Safe agents can fail together.
Terminology
Abstract
Safe agents can fail together. Multi-agent LLM systems (MAS) move information, state, decisions, and authority across principal boundaries, creating failures that local checks may miss. Without an execution-level view, a multi-agent setting can easily be mistaken for evidence of a genuinely multi-agent security effect. We thus systematize MAS security through an execution-centered analysis of 197 works, covering six interaction interfaces, four adversary positions, seven system-level risks, and eight recurring attack paths. We introduce an A-I-R framework that organizes attacks by adversary position, interaction interface, and resulting system-level risk, unifying otherwise fragmented attack mechanisms across MAS. We organize defenses through a five-part contract covering path target, observation, intervention, trust boundary, and recovery, and identify path closure and recovery as key challenges. We audit 44 evaluation and benchmark works and identify open challenges in isolating interaction effects, designing comparable and diagnostic metrics, supporting reuse across MAS designs, and evaluating open-system operation. Together, these findings motivate an interaction-aware view of MAS security: trace attacks end to end, test whether defenses close those paths, and evaluate system-level effects with appropriate counterfactuals.
Sources
- LCGuard: Latent Communication Guard for Safe KV Sharing in Multi-Agent Systems
- SNEAK: Evaluating Strategic Communication and Information Leakage in Large Language Models
- Many-to-One Adversarial Consensus: Exposing Multi-Agent Collusion Risks in AI-Based Healthcare
- Institutional AI: Governing LLM Collusion in Multi-Agent Cournot Markets via Public Governance Graphs
- When Latent Agents Lie: KV-Cache Integrity in Multi-Agent LLM Collaboration
- Cross-Layer Semantic Flow Reconstruction for Attack Detection in Agentic Systems
- When Child Inherits: Modeling and Exploiting Subagent Spawn in Multi-Agent Networks
- Byzantine Fault-Tolerant Multi-Agent System for Healthcare: A Gossip Protocol Approach to Secure Medical Message Propagation
- When Persuasion Overrides Truth in Multi-Agent LLM Debates: Introducing a Confidence-Weighted Persuasion Override Rate (CW-POR)
- AgentMonitor: A Plug-and-Play Framework for Predictive and Secure Multi-Agent Systems
- Prompt Injection Mitigation with Agentic AI, Nested Learning, and AI Sustainability via Semantic Caching
- Agentic JWT: A Secure Delegation Protocol for Autonomous AI Agents
- Convergence dynamics of Agent-to-Agent Interactions with Misaligned objectives
- Architecture Matters for Multi-Agent Security
- Maris: A Formally Verifiable Privacy Policy Enforcement Paradigm for Multi-Agent Collaboration Systems
- MAD-Spear: A Conformity-Driven Prompt Injection Attack on Multi-Agent Debate Systems
- Byzantine Cheap Talk: Adversarial Resilience and Topology Effects in LLM Coordination Games
- A Survey of LLM-Driven AI Agent Communication: Protocols, Security Risks, and Defense Countermeasures
- Collective Hallucination in Multi-Agent LLMs:Modeling and Defense
- Architecture Matters: Comparing RAG Systems under Knowledge Base Poisoning
Related papers
- SoK: AI-Augmented Binary Reversing
- Relaxed Sender Anonymity for CBDC Interbank Settlement: A Zero-Knowledge Approach on Permissioned EVM
- Calibration-Family Overfit: Why Trusted Sabotage Monitors Don't Transfer Across Lineages
- Estimating the Social Cost of Corporate Data Breaches
- Efficient Fuzzy PSI under One-Sided Assumptions
- Sealing the Audit-Runtime Gap for LLM Skills