TabAgent: Distilling LLM Agent Components into Cheap Classifiers

The gist: TABAGENT proposes a framework for replacing agentic generative components in closed-set selection tasks with a compact textual-tabular classifier trained on execution traces, enabling agents to optimize their own decision-making by learning from successful executions.

How it works

TABAGENT is a general framework that replaces generative decision components in closedset selection tasks with a compact textual-tabular classifier trained on signals extracted from execution traces, by (i) automatically distilling schema/state/dependency features from trajectories TabSchema, (ii) augmenting coverage with schemaaligned synthetic supervision TabSynth, and (iii) scoring candidates with a compact textual tabular classifier TabHead. This approach enables agentic systems to automatically optimize their own decision-making by learning from successful executions, allowing for seamless replacement of expensive generative components.

TabSchema: Agentic Feature Extraction

TABSchema employs a hierarchical agentic orchestration to extract behavior-grounded features from agent trajectories, ensuring both comprehensive coverage and explainable feature selection through multi-stage validation and judicial review. The FeatureOrchestrator coordinates agents and maintains a ledger while routing a trajectory batch T into the FeatureAnalyzer, which first designs feature specifications before any code is written. The analyzer enumerates (i) extractable signals lifted directly from traces (e.g., thought snippets from a specific node, last return status, error types), and (ii) computed signals derived from trace structure (e.g., step index within the agentic loop, “has agent X been visited?”, co-usage windows). This process includes a multi-LLM judicial review where three specialized judges independently read the features card and return a score in [0, 1] with short justification.

TabSynth: Synthetic data generation

To close the gap left by real trajectories under rare but decision critical patterns, TABSynth introduces a schema-aligned augmentation module that expands supervision while preserving the behavior-grounded signals surfaced by TabSchema. This module consumes the feature cards produced by TabSchema and instantiates additional training rows for each pair (context, candidate) constrained by three inputs: the FEATURES CARD schema, TRAJECTORY TABLE providing observed execution patterns, and TOOL CATALOG SUMMARY specifying taxonomy and argument as present in real data. Each synthetic row is a feature vector with populated features fields, validated by (i) schema checks (types, arity, taxonomy level), (ii) dependency feasibility (no impossible chains), and (iii) deduplication/diversity against real data.

TabHead: Textual–Tabular Classification

TABHead instantiates a SOTA textual–tabular classifier over two aligned views of each candidate. The tabular view is the TabSchema feature vector, and the candidate textual representation is a single string formed by concatenating the candidate name with its description and argument hints. The training phase mixes real trajectories with schema-aligned TabSynth data to widen coverage while staying within the schema. For ranking, it scores all candidates by p(y x) and returns the top-K results.

Experimental Validation and Results

Experiments evaluate TABAGENT on AppWorld, maintaining task-level success while eliminating shortlist-time LLM calls and reducing latency (by ∼ 95%) and inference cost (by 85–91%). Across five applications, TabAgent (+synth) leads or ties the strongest baseline, attaining Recall@7 ≥ 0.88 in every domain and Recall@9 ≥ 0.92 when narrowing catalogs of hundreds of tools to just nine candidates. The framework successfully reproduces the toolshortlisting decision boundaries of multi-call LLM controllers while being 95% faster and 85–91% cheaper than CUGA’s original shortlister.

Key Contributions

The work makes three key contributions: (1) TabAgent First framework to abstract agentic decision heads into a tabular representation and systematically replace generative modules with a compact classifier. (2) Automatic schema engineering Extraction of schema, state, and dependency signals from trajectories and tool I/O, together with schema-aligned synthetic supervision that removes manual labels and enables rapid adaptation. (3) Production-scale validation In IBM CUGA (a stateof-the-art open-source system), replacing only the tool shortlisting generative component preserves shortlist quality while removing shortlist-time LLM calls and yielding substantial latency and cost reductions.

Impact Statements

The primary positive impact is improved deployability of agentic systems through substantially reduced inference latency and cost, which can broaden access to practical agents and reduce computational and energy overhead in production settings. This approach establishes a general paradigm for autonomous agentic optimization by allowing systems to continuously improve their own efficiency by learning from successful execution patterns and performing hot-swaps of optimized components.

Constraints and Future Work

Several constraints limit this work’s scope, including the need for sufficient execution logs from reliable agentic systems, limited public availability of state-of-the-art agents, and domain expertise required for optimal feature engineering. Future work includes automatic detection mechanisms that identify which generative components in existing agent architectures can be replaced with tabular discriminative heads by analyzing execution traces for stable input and output patterns. The framework establishes a pathway toward self-optimizing agent architectures that automatically reduce computational overhead while maintaining task performance.

Error Analysis

An error analysis revealed three primary categories of discrepancies in tool shortlisting, with 93% of observed errors (categories 1 and 2) having minimal to low downstream impact on task success. These include optional or nice-to-have tools (62% of errors), semantically similar endpoint confusion (31% of errors), and genuine required-tool omissions (7% of errors). The framework succeeds in this objective, preserving the core shortlisting capability while introducing a small tail of correctable errors amenable to targeted fixing.

Cross-Model Validation

Robustness experiments using GPT-4.1, Gemini 2.5, and Claude 4.5 Sonnet demonstrated that TabSchema extracts genuine decision-relevant structure from execution traces rather than model-specific artifacts. The three models produced feature sets of varying size with substantial overlap (Table 14), suggesting that state and dependency indicators dominate the predictive mass regardless of which model extracted them. This validation supports the claim that TabSchema extracts genuine decision-relevant structure from execution traces rather than model-specific artifacts, evidenced by 69 percent three-way agreement and 0.81 mean semantic similarity.

Cost and Efficiency

The efficiency gap widens as models grow, with the TabAgent head (≈ 50M parameters) running in 2.682 ms at a cost of approximately 38,000× to 142,000× higher than LLM variants. This demonstrates that scaling general-purpose LLMs does not resolve the efficiency constraint without sacrificing deployment practicality. The results show TabAgent in the lower-left “faster & cheaper” corner of the runtime–cost plane, while all LLM points cluster orders of magnitude away.

Conclusion

TABAGENT introduces a novel tabular representation framework for agent decision-making that enables single-pass discriminative alternatives to agentic generative AI components. The framework transforms execution traces into structured schema/state/dependency features (TabSchema), augments coverage with schema-aligned synthesis (TabSynth), and trains a compact TabHead that ingests these features. Our central hypothesis is that execution traces contain sufficient structural signals for a calibrated tabular classification head (50M parameters) to make setcomplete decisions that replicate the boundaries of generative components <ref:2602.

1. Replace expensive generative decision components like tool shortlisting with a compact textual tabular classifier TabHead to reduce latency by about 95% and inference cost by 85-91%.

2. Implement TabSchema to automatically distill schema, state, and dependency features from execution traces, ensuring that the classifier is trained on behavior-grounded signals rather than just task descriptions.

3. Augment coverage using TabSynth to generate schema-aligned synthetic supervision for the classifier, which expands its ability to handle low-frequency schema and dependency patterns.

4. Enable continuous self-optimization where agentic systems can accumulate successful traces, extract behavioral signals, train compact classifier for generative bottlenecks, and perform hot-swaps during operation without manual intervention.

5. Create a general paradigm for learned discriminative replacements of generative bottlenecks in production agentic architectures by binding the framework only to execution traces and typed schemas.