SkillSpec: Consensus-Gated Agent Skill Evolution via Representation Specialization

The gist The introduction identifies a key distinction between learning what knowledge to retain and determining how to organize it, leading to SkillSpec, a two-phase framework comprising consensus-gated evolution and representation specialization.

SkillSpec Framework

SkillSpec is introduced as a two-phase framework designed for skill optimization without updating model weights The introduction identifies a key distinction between learning what knowledge to retain and determining how to organize it, leading to SkillSpec, a two-phase framework comprising consensus-gated evolution and representation specialization In the consensus-gated phase, complementary editing intents generate complete candidate skills An update is committed only when paired evaluations reach consensus, requiring sufficient overall improvement and non-negative aggregate paired gain in every repeated evaluation

Phase I: Consensus-Gated Skill Evolution

Phase I evolves the initial textual artifact by using a multi-intent candidate generation process combined with repeated paired validation The framework generates candidates under four complementary editing intents: repair, preserve, simplify, and rewrite The screening process compares each candidate with the current skill on the same validation instances to compute paired counts for improvements and regressions An update is committed only when it improves performance overall with non-negative aggregate paired gain in every repeated evaluation

Phase II: Representation Specialization

Phase II focuses on selecting a representation based on signals derived from the optimization trajectory, specifically process sensitivity (PSS) and redundancy sensitivity (RSS) Phase II estimates structural sensitivity from telemetry and uses these signals to select a flat, graph, or hybrid representation The selection is guided by Table 1, where the router chooses between flat if process sensitivity is low and graph if both process and redundancy sensitivities are high This phase preserves the validated backbone as an immutable source while specializing a derived artifact to obtain A⋆

Evaluation and Results

SkillSpec was evaluated on six benchmarks across three frontier language models, showing improvements over SkillOpt by 6.89% averaged across the three models Across the six benchmarks, SkillSpec improves average success rate over the no-skill baseline by up to 24.59% For instance, on GPT–4.1, SkillSpec yields gains of 11.63%, 5.92%, and 3.12% compared to SkillOpt for GPT–4.1, GPT–5.4 Nano, and GPT–5.4 respectively The results demonstrate that reliable skill evolution and representation specialization address complementary objectives: deciding what knowledge to retain and how to structure it for inference

Representation Specialization Dynamics

The selection of the representation is guided by comparing predicted gains under different structural conditions, such as flat, graph, or hybrid structures Process sensitivity measures dependence on ordered steps, while redundancy sensitivity measures vulnerability to duplicated guidance The selector first uses PSS (ψ) to determine whether explicit ordering is warranted and then uses RSS (ρ) to distinguish between graph and hybrid representations For example, for GPT–5.4 Nano, the low-PSS, high-RSS profile suggests limited support for beneficial explicit ordering The final results show that SkillSpec achieves the highest average success rate for all three evaluated models across six benchmarks It is also observed that Phase II yields additional gains on average through representation-specific continued optimization

Conclusion

SkillSpec introduces a two-phase framework for improving LLM agents through skill optimization without updating model weights Phase I uses consensus-gated evolution to refine reusable knowledge, while Phase II uses trajectory-derived sensitivity signals to select and optimize a flat, graph, or hybrid organization These findings suggest that effective skill optimization requires attention not only to what knowledge an agent retains, but also to how that knowledge is structured for execution The framework successfully improves performance across diverse tasks by decoupling content evolution from representation structure selection The overall approach demonstrates that explicit skill evolution and reorganization can remain valuable even when the underlying model is less capable

--- Page 1 ---

The gist The introduction identifies a key distinction between learning what knowledge to retain and determining how to organize it, leading to SkillSpec, a two-phase framework comprising consensus-gated evolution and representation specialization.

SkillSpec Framework

SkillSpec is introduced as a two-phase framework designed for skill optimization without updating model weights In the consensus-gated phase, complementary editing intents generate complete candidate skills An update is committed only when paired evaluations reach consensus, requiring sufficient overall improvement and non-negative aggregate paired gain in every repeated evaluation

Representation Specialization Dynamics

The selection of the representation is guided by comparing predicted gains under different structural conditions, such as flat, graph, or hybrid structures Process sensitivity measures dependence on ordered steps, while redundancy sensitivity measures vulnerability to duplicated guidance The selector first uses PSS (ψ) to determine whether explicit ordering is warranted and then uses RSS (ρ) to distinguish between graph and hybrid representations For example, for GPT–5.

1. Bold skill evolution through consensus-gated validation: An update is committed only when paired evaluations reach consensus, requiring sufficient overall improvement and non-negative aggregate paired gain in every repeated evaluation. This ensures that retained knowledge is based on reliable execution evidence rather than aggregate scores that may obscure an edit’s behavioral effects.

2. Structural representation specialization: Phase II estimates process and redundancy sensitivity, then uses these signals to select a flat, graph, or hybrid representation. This allows the system to choose between structures based on learned needs, as shown in Table 7 where dependencies between spreadsheet operations or mathematical reasoning motivate an ordered core, while redundancy sensitivity guides the use of independently retrievable units.

3. Multi-intent candidate generation: The framework generates candidates under four intents: repair, preserve, simplify, and rewrite. This allows the system to address different optimization goals simultaneously, enabling it to choose which edits are most beneficial based on the specific task context.

4. Trajectory-guided specialization: The process uses evidence from rejected candidates to inform representation selection: Building on the validated skill and the trajectory of candidate evaluations, the second phase estimates process and redundancy sensitivity. This ensures that signals of process and redundancy sensitivity derived from the full optimization trajectory, including accepted and rejected candidates, guide the selection of a flat, graph, or hybrid representation.

5. Adaptive representation selection: The system selects a structure based on learned sensitivities: PSS (ψ) therefore measures evidence that explicit process structure improves candidate outcomes, while RSS (ρ) measures evidence that suppressing interference improves them. This guides the choice among representations to match the task's need for either explicit dependency structure or modularity against duplicated, conflicting, or excessive guidance.

6. Specialized procedural content: The system can learn complex organizational rules: Distinguish theorem-style statements with subtle strength differences, which is a specialization for LiveMath, demonstrating how the learned representation can capture fine-grained procedural needs beyond simple text editing.