Dream2Discern: Learning from Hard Latent Variants for Runtime Failure Prediction in Multi-Agent Systems
cs.LG, cs.AI, cs.CL, cs.MA
Submitted: 2026-10-12
Updated: 2026-10-12
Comments: 27 pages, 6 figures
Code: https://github.com/ZBox1005/Dream2Discern
Project page: https://zbox1005.github.io/dream2discern-project
License: http://creativecommons.org/licenses/by/4.0/
The gist: Reliable deployment of LLM-based multi-agent systems calls for runtime auditors that predict terminal outcomes from observed execution prefixes.
Terminology
Abstract
Reliable deployment of LLM-based multi-agent systems calls for runtime auditors that predict terminal outcomes from observed execution prefixes. These auditors typically learn from a limited corpus of trajectories with verified outcomes. Expanding this corpus requires costly execution and verification, yet broader failure coverage alone provides no explicit criterion for identifying prefixes that the current auditor finds difficult to classify. To this end, we propose Dream2Discern, a runtime auditor trained through implicit augmentation guided by its own prediction loss. During Dream, we select perturbations of the auditor's representation operators to increase prediction loss on recorded prefixes, yielding hard latent variants. During Discern, the auditor learns to predict execution outcomes from these variants using the original labels. The variants are regenerated as the auditor learns, turning a limited corpus into a renewable source of hard examples and requiring no additional trajectory collection or outcome verification. Extensive evaluations on AFTraj-2K and StepShield show that Dream2Discern outperforms competitive failure auditors and achieves over 90% trajectory-level weighted accuracy on both benchmarks while balancing prediction accuracy and timeliness. Project page: https://zbox1005.github.io/dream2discern-project/
Sources
- When Evidence is Sparse: Weakly Supervised Early Failure Alerting in Dialogs and LLM-Agent Trajectories
- CausalFlow: Causal Attribution and Counterfactual Repair for LLM Agent Failures
- StepShield: When, Not Whether to Intervene on Rogue Agents
- Magentic-One: A Generalist Multi-Agent System for Solving Complex Tasks
- The Llama 3 Herd of Models
- LoRA: Low-Rank Adaptation of Large Language Models
- PrefixGuard: Online Failure Warning and Trace-Grounded Diagnosis for LLM Agents
- Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations
- TRACES: Proactive Safety Auditing for Multi-Turn LLM Agents via Trajectory-State Modeling
- DeepInception: Hypnotize Large Language Model to Be Jailbreaker
- AgentDoG: A Diagnostic Guardrail Framework for AI Agent Safety and Security
- Uncertainty Estimation in Autoregressive Structured Prediction
- Hide-and-Seek in Trajectories: Discovering Failure Signals for VLA Runtime Monitoring
- The Why Behind the Action: Unveiling Internal Drivers via Agentic Attribution
- Out-of-Distribution Detection and Selective Generation for Conditional Language Models
- Rethinking How to Remember: Beyond Atomic Facts in Lifelong LLM Agent Memory
- Max-Rank: Efficient Multiple Testing for Conformal Prediction
- Out-of-distribution Detection with Implicit Outlier Transformation
- Qwen3 Technical Report
- Tracing Agentic Failure from the Flow of Success
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks