TopoCurate:Modeling Interaction Topology for Tool-Use Agent Training
cs.LG, cs.CL
Submitted: 2026-03-02
Updated: 2026-10-13
Comments: arxiv version of EMNLP2026
License: http://creativecommons.org/licenses/by/4.0/
The gist: Training tool-use agents typically relies on outcome-based filtering: Supervised Fine-Tuning (SFT) on successful trajectories and Reinforcement Learning (RL) on pass-rate-selected tasks.
Terminology
Abstract
Training tool-use agents typically relies on outcome-based filtering: Supervised Fine-Tuning (SFT) on successful trajectories and Reinforcement Learning (RL) on pass-rate-selected tasks. However, this paradigm ignores interaction dynamics: successful trajectories may lack error recovery or exhibit redundancy, while pass rates fail to distinguish structurally informative tasks from trivial ones. We propose TopoCurate, an interaction-aware framework that projects multi-trial rollouts from the same task into a unified semantic quotient topology. By merging equivalent action-observation states, this projection transforms scattered linear trajectories into a structured manifold that explicitly captures how tool invocations and environmental responses drive the divergence between effective strategies and failure modes. Leveraging this representation, we introduce a dual-selection mechanism: for SFT, we prioritize trajectories demonstrating reflective recovery, semantic efficiency, and strategic diversity to mitigate covariate shift and mode collapse; for RL, we select tasks with high error branch ratios and strategic heterogeneity, maximizing gradient Signal-to-Noise Ratio to address vanishing signals in sparse-reward settings. Evaluations on BFCLv3 and Tau2 Bench show that TopoCurate achieves consistent gains of 4.2% (SFT) and 6.9% (RL) over state-of-the-art baselines. We will release the code and data soon for further investigations.
Sources
- $\tau^2$-Bench: Evaluating Conversational Agents in a Dual-Control Environment
- AutoForge: Automated Environment Synthesis for Agentic Reinforcement Learning
- Pushing the Boundaries of Natural Reasoning: Interleaved Bonus from Formal-Logic Verification
- ACEBench: Who Wins the Match Point in Tool Usage?
- Reinforcement Learning for Tool-Integrated Interleaved Thinking towards Cross-Domain Generalization
- Rethinking LLM Training through Information Geometry and Quantum Metrics
- WebOperator: Action-Aware Tree Search for Autonomous Agents in Web Environment
- Towards General Agentic Intelligence via Environment Scaling
- Interaction as Intelligence Part II: Asynchronous Human-Agent Rollout for Long-Horizon Task Training
- Interaction Dynamics as a Reward Signal for LLMs
- Jina Embeddings 2: 8192-Token General-Purpose Text Embeddings for Long Documents
- Tree Search for LLM Agent Reinforcement Learning
- Exploring Expert Failures Improves LLM Agent Tuning
- ToolPRMBench: Evaluating and Advancing Process Reward Models for Tool-using Agents
- TreePO: Bridging the Gap of Policy Optimization and Efficacy and Inference Efficiency with Heuristic Tree-based Modeling
- Simulating Environments with Reasoning Models for Agent Training
- ToolPRM: Fine-Grained Inference Scaling of Structured Outputs for Function Calling
- DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models
- APIGen-MT: Agentic Pipeline for Multi-Turn Data Generation via Simulated Agent-Human Interplay
- CARL: Criticality-Aware Agentic Reinforcement Learning
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks