AgentTune: A Unified Training Lifecycle for Tool-Using LLM Agents

arXiv:2610.13335 · cs.LG · Submitted 2026-10-09 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "AgentTune: A Unified Training Lifecycle for Tool-Using LLM Agents".

Jane: The gist: AgentTune provides a unified framework, AgentTune introduces a two-tier EventLog representation that preserves agent trajectories while supporting consistent projections for training, evaluation, and auditing.

Tom: First, who's behind it and why it matters.

Paper summary: Tom: So we’re talking about AgentTune again. The paper says they organize the library around this one interface between all those stages, which is that two-tier EventLog representation. They show how agent trajectories from the runtime, reinforcement learning rollouts, DECIDE workflows and evaluation traces all project into this schema.

Jane: And crucially, they show that the same record then projects out to things like SFT rows for training and evaluator inputs and even audit records for auditing. This consistency is what makes it powerful.

Lu: They emphasize that the library is designed to support the entire agent development lifecycle from build and collection through evaluation, training, distillation and recovery, but they evaluate these capabilities separately rather than claiming one single experiment proves the whole thing works end to end.

Meng: They point out that this is not a claim of methodological equivalence for everything; it’s more about a complementary relationship between the pieces. The spine and DECIDE share the reinforcement learning core and the EventLog boundary, but they are kept separate so neither drags the other's execution machinery into something it doesn't need.

Lalam: That sounds like they’re keeping things modular, which is good for production. They also highlight that the legal case study provides what they call the strongest task-level evidence for this approach.

Tom: And that legal case study involves a search agent being trained on synthesized questions and then evaluated on three unseen corpora, which they say shows the real value of this unified trajectory contract.

Jane: So, before we talk about how they actually do it, we need to understand what the architecture looks like. This paper makes four main contributions that define what AgentTune is supposed to deliver.

Lu: First is a unified trajectory contract, which means that two-tier EventLog represents agent interactions once and provides projections for training, evaluation and audit including segment-aware completion-only loss masking.

Meng: Second they have a shared agent training stack where the agentic spine and the TRL backed reinforcement learning core provide a common runtime and training interface for tool-using agents across GRPO PPO DPO RLOO and BCO.

Lalam: And third, decision workflows and recovery, where DECIDE provides YAML defined workflows with audit trails that support converting detected failures into corrective training data and gated retraining or deployment.

Tom: And the fourth contribution is empirical evidence across representative paths, GPU verified smoke tests establish that the principal training paths execute end to end while the legal contract search study provides what they say is the strongest task-level result.

Jane: So, to summarize this part of AgentTune: it’s about giving agents a single language for everything from how you train them to how you fix them, all channeled through that EventLog structure.

Lu: This unified trajectory contract is the foundation that allows all these different capabilities to plug into each other without needing separate representations or pipelines.

Conclusion: Tom: So we’ve gone over the paper "AgentTune: A Unified Training Lifecycle for Tool-Using LLM Agents." The core idea is this single trajectory schema projecting in from RL rollouts DECIDE runs and evaluation traces, and projecting out to SFT rows evaluator inputs and audit records.

Jane: It seems the authors are not claiming one experiment demonstrates the entire lifecycle, but they do show how these capabilities can be combined through composable interfaces. They build an agentic spine on top of that foundation.

Lu: The implication here is that you don't have to design a totally separate pipeline for every single thing you want to do with an agent—training or evaluation or recovery. You just plug it into this shared interface and it works across different methods.

Meng: From a deployment standpoint, the self-healing closed loop based on DECIDE workflows and audit trails is what really catches my eye when I think about shipping models to production, because it turns failures into corrective training data in a structured way.

Lalam: I find that the empirical evidence from the legal contract search study is what gives this paper real weight, showing that this unified approach works well across different tasks.

Tom: So for the listener who just wants the plain idea: AgentTune provides one trajectory schema for everything—from how you train a tool-using agent to how you recover it if it breaks. It’s about making the lifecycle of building these agents more consistent and less fragmented than what we see today.

R E Zera Marveen Lyngkhoi, Abhivansh Gupta, Vidushee Vats, Ram Mohan Rao Kadiyala, Vinay Kumar Sankarapu Pratinav Seth

Lexsi Labs

cs.LG

Submitted: 2026-10-09

Updated: 2026-10-09

Code: https://github.com/Lexsi-Labs/AgentTune

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 92/100

The gist: The gist: AgentTune provides a unified framework, AgentTune introduces a two-tier EventLog representation that preserves agent trajectories while supporting consistent projections for training,

Key concepts

Two-tier EventLog: This is the core data structure that captures every step of an agent's journey. The light tier records what happened (like a tool call or observation), while the full tier adds detailed information such as token spans and probabilities. This unified log serves as the single source of truth for all subsequent processes, ensuring consistency between different stages.
Agentic Spine: The spine is the operational layer where an agent strategy interacts with a harness to execute tasks. It is responsible for logging every step of the process into the EventLog. This component drives the agent's actual behavior and serves as the starting point for collecting trajectories needed for training and auditing.
DECIDE Workflows: DECIDE compiles structured, YAML-defined decision workflows that are appended to an audit trail. These workflows allow agents to be designed with explicit decision-making logic. Crucially, they enable a closed-loop recovery system where detected failures can trigger corrective training data and gated retraining.
Segment-aware Loss Masking: This technique converts a full agent trajectory into exactly one training row. It specifically masks the loss calculation so that only tokens generated by the agent itself are used to calculate the model's learning objective, ignoring tokens from the environment or tools.

Summary

The gist: AgentTune provides a unified framework, AgentTune introduces a two-tier EventLog representation that preserves agent trajectories while supporting consistent projections for training, evaluation, and auditing.

How it works

AgentTune is organized around one interface between these stages: the two-tier EventLog. Agent trajectories from the runtime, RL rollouts, DECIDE workflows, and evaluation traces project into the schema; the same record projects out to SFT rows, evaluator inputs, and audit records<ref:2610.13335#pg5>. The library is designed to support the agent development lifecycle from build and collection through evaluation, training, distillation, and recovery<ref:2610.13335#pg3>.

The schema is one standard-library module (agenttune.agentic.events). An EventLog is an ordered list of Events plus a tier<ref:2610.13335#pg5>. Each Event carries an EventKind, a free-form payload dict, and optional token span, logprobs, and scope fields<ref:2610.13335#pg5>. Eight kinds cover the whole trajectory: model text and reasoning, a tool call and its result, an environment observation, a turn boundary, a reward, and a memory operation<ref:2610.13335#pg5>.

Unified Training Lifecycle

The library is designed to support the agent development lifecycle from build and collection through evaluation, training, distillation, and recovery<ref:2610.13335#pg3>. The common trajectory substrate is therefore complementary rather than a claim of methodological equivalence<ref:2610.13335#pg9>.

The spine and DECIDE share the RL core and the EventLog/audit boundary<ref:2610.13335#pg5>. The remaining coupling is onedirectional and deliberately thin: DECIDE never imports the spine, and the spine imports only DECIDE’s standard-library dataclass contracts, so neither subsystem drags the other’s execution machinery into a process that does not need it<ref:2610.13335#pg5>.

Key Components and Capabilities

AgentTune makes four contributions:

A unified trajectory contract. The two-tier EventLog represents agent interactions once and provides projections for training, evaluation, and audit, including segment-aware completion-only loss masking

A shared agent-training stack. The agentic spine and TRL-backed RL core provide a common runtime and training interface for tool-using agents across GRPO, PPO, DPO, RLOO, and BCO

Decision workflows and recovery. DECIDE provides YAML-defined workflows and audit trails, while the architecture supports converting detected failures into corrective training data and gated retraining/deployment

Empirical evidence across representative paths. GPU-verified smoke tests establish that the principal training paths execute end to end, while the legal contract-search study provides the strongest task-level result

Core Mechanisms

Two tiers: A light-tier log is observational: it carries the content of a trajectory (what tool was called, what came back, what the reward was) and is produced by a harness-driven episode, a DECIDE run, or an evaluation trace<ref:2610.13335#pg5>. A full-tier log also carries token spans and logprobs and comes only from a rollout engine<ref:2610.13335#pg5>.
Segment-aware, completion-only loss: as dataset rows(sft) turns one full-tier log into exactly one training row<ref:2610.13335#pg5>. The formal objective for parameters θ, query q, and generated tokens y is L(θ) = −∑ i∈Iagent wi log πθ(yi y<i, q), where Iagent indexes tokens the agent itself generated and wi ∈ 0, 1, 2 is the segment weight of the message containing token i; environment tokens have wi = 0<ref:2610.13335#pg6>.
Agentic spine: The spine is the build-and-run layer: an AgentStrategy plus a Harness, driven by a Project that logs every step to the EventLog<ref:2610.13335#pg5>.
DECIDE workflows and recovery: DECIDE, implemented in agenttune.decide, compiles YAML decision workflows, appends each run to audit.jsonl, and uses that audit trail for closed-loop recovery<ref:2610.13335#pg6>.

Decision Workflows and Self-Healing

The self-healing closed loop reads the audit log (or spine-collected trajectories, via Project.heal) and runs two stages connected by a thread-safe FIFO-capped TrainingBuffer<ref:2610.13335#pg13>. Stage 1 turns a failure into an example<ref:2610.13335#pg10>. Stage 2 decides whether to retrain and whether to ship<ref:2610.13335#pg10>.

Empirical Evidence

The legal case study provides the strongest task-level evidence: a search agent is trained on synthesized questions and then evaluated on three unseen corpora (§5.1)<ref:2610.13335#pg9>. GRPO improves held-out CUAD exact match from 60.0% to 69.0% and reduces searches per question from 3.00 to 2.90<ref:2610.13335#pg18>.

Use Case

The legal contract search agent is trained on a raw corpus, through synthesized training data and a trained search policy, to a distilled student small enough to run on-premises<ref:2610.13335#pg6>. This demonstrates the library's capability across the entire lifecycle<ref:2610.13335#pg9>.

Conclusion

AgentTune is designed to support the agent development lifecycle through one trajectory schema rather than separate representations for execution, training, evaluation, and recovery<ref:2610.13335#pg9>. The EventLog projects in from RL rollouts, DECIDE runs, and evaluation traces and out to training rows, evaluator inputs, and audit records<ref:2610.13335#pg5>. Around this contract, the library provides an agentic runtime, a TRL-backed RL core with multi-turn tool-calling rollouts, agentic distillation and RAG synthesis, YAML decision workflows, and a designed self-healing path<ref:2610.13335#pg9>. The reported experiments establish representative components separately; the legal case study provides the strongest task-level evidence<ref:2610.13335#pg9>.

Future Work

Several directions are open. Multilingual and non-contractual domains need verification, and naturally occurring user queries remain to be tested<ref:2610.13335#pg23>. The standalone core/sft path is not wired end to end, and a generated API reference would replace the current hand-maintained one<ref:2610.13335#pg30>. Cluster-scale RL, which AgentTune does not attempt, is left to verl and OpenRLHF<ref:2610.13335#pg9>. The most valuable single addition is external calibration: one public tool-use benchmark scored through create agentic trainer, which would place this paper’s internal numbers on a scale a reader already knows<ref:2610.13335#pg30>.

Security and Ethical Considerations

The shipped defaults favor usability over confinement<ref:2610.13335#pg25>. The closed loop closes over its own outputs: failures are classified, turned into preference pairs, and fed back as training data<ref:2610.13335#pg10>. Audit records are written verbatim and are not redacted<ref:2610.13335#pg25>. The BFSI KYC example in Listing 10 carries name, date of birth, employer and income through the audit trail and into the retraining buffer<ref:2610.13335#pg25>.

Acknowledgments

AgentTune builds on HuggingFace Transformers, TRL, HuggingFace Datasets, LangGraph, and OpenEnv<ref:2610.13335#pg28>. The repository ships scripted spine case studies for each of these domains, run with open models and open data<ref:2610.13335#pg9>.

References

[1] Leandro von Werra, Younes Belkada, Lewis Tunstall, Edward Beeching, Tristan Thrush, Nathan Lambert, and Shengyi Huang. TRL: Transformer reinforcement learning. https://github.com/huggingface/trl, 2020<ref:2610.

Improvements for AI systems

AgentTune provides A unified trajectory contract by representing agent interactions once and providing projections for training, evaluation, and audit, ensuring consistency across all stages of development. This enables a shared agent-training stack that supports GRPO, PPO, DPO, RLOO, and BCO, allowing researchers to combine strategies without separate pipelines.
Implement a closed loop system using DECIDE workflows to automate self-healing: the system can use the FailureDetector and FailureClassifier to turn failures into corrective training data via a TrainingExampleGenerator. This ensures that production failures are systematically converted into preference data for retraining, governed by criteria such as T1 total failures exceeded buffer size ≥ threshold.
Develop agents capable of learning efficient search policies by integrating retrieval with reinforcement learning: the system supports Search-R1 [51] and FrugalRAG [43], allowing policies to learn when to retrieve rather than consuming fixed context, which improves performance metrics like Searches per question from 3.00 to 2.90.
Create highly specialized, distilled models for specific tasks: use Agentic distillation where the student is trained on full-tier trajectories but with a segment-aware completion-only loss, allowing the student to learn the teacher’s procedure and not its retrieved text. This results in a student model that can run on premises without external inference calls.
Build agents with robust decision-making capabilities by compiling YAML workflows into LangGraph state graphs via DECIDE: this enables complex, branching logic like human review filesystem handshake or parallel fan-out to independent branches, ensuring the agent's control flow is auditable and reproducible.