BLADE: Auditing Information Use for Failure Attribution in LLM Multi-Agent Systems
cs.AI, cs.CL
Submitted: 2026-10-13
Updated: 2026-10-13
Comments: 16 pages, 6 figures, 3 tables
Code: https://github.com/GXG-CS/BLADE
License: http://creativecommons.org/licenses/by/4.0/
The gist: Large language model multi-agent systems extend individual agents' capabilities through collaboration across specialized roles, tools, and workflows.
Terminology
Abstract
Large language model multi-agent systems extend individual agents' capabilities through collaboration across specialized roles, tools, and workflows. Yet collaboration makes failures harder to diagnose: one agent may misinterpret a correct result while others carry the mistake forward. Failure attribution therefore seeks to identify the responsible agent and decisive error step. Recent methods trace dependencies, but knowing where information came from is not the same as knowing where it went wrong. We introduce BLADE (Blame Localization via Audited Dependency Evidence), a training-free framework that moves beyond tracing information to auditing its use. Using Jev for structured probabilistic judgments, BLADE uses dependency probabilities to select relevant source-decision handoffs for auditing. At these handoffs, it examines whether receiving decisions use their sources faithfully and within task requirements, distinguishing error introduction, propagation, and repair. These findings inform a global diagnosis that considers all execution steps. Without reference answers, BLADE achieves the best or tied-best step accuracy among eight strategies under a shared Jev decision interface on 1,228 failed runs across five Who&When and Who&When Pro subsets, outperforming All-at-once by 2.2-11.1 percentage points. Making responsibility evidence explicit offers a path toward more interpretable and reliable agentic systems.
Sources
- AgentRx: Diagnosing AI Agent Failures from Execution Trajectories
- DeFA: Dependency-Guided Failure Attribution for LLM Agents
- Magentic-One: A Generalist Multi-Agent System for Solving Complex Tasks
- Who&When Pro: Can LLMs Really Attribute Failures in AI Agents?
- TRAJDEBUG: Tracing Error Lifecycle to Identify Critical Failures in Long-Horizon Agent Trajectories
- FALAT: Tracing Failures in LLM Agent Trajectories via Dependency-Guided Search
- Adaptive In-conversation Team Building for Language Model Agents
- From Flat Logs to Causal Graphs: Hierarchical Failure Attribution for LLM-based Multi-Agent Systems
- DCFA: Dual-view Causal-inspired Attribution for Failure Reasoning in LLM-based Multi-agent Systems
- AgenTracer: Who Is Inducing Failure in the LLM Agentic Systems?
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems