Skill-V: Verifiable Self-Evolving Skill Library for Interactive Agents

arXiv:2610.11781 · cs.CV · Submitted 2026-10-08 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "Skill-V: Verifiable Self-Evolving Skill Library for Interactive Agents".

Jane: The gist The introduction states that reliable skill evolution requires more than accumulating experience; it must learn which knowledge to retain, when to revise it,

Tom: First, who's behind it and why it matters.

Paper summary: Tom: So, looking at the whole Skill-V concept and what we just covered about how it uses failures to add skills and disagreements to revise them, the authors are really pushing this idea of verifiable evolution Tom. It’s about making sure that when an agent learns something new or changes something old, there's a clear, testable reason for the change Tom.

Jane: And they present this Skill-V system as a way to make agents more reliable because it actively maintains the quality of their stored knowledge instead of just letting it pile up Jane. It moves learning from being purely about accumulation to being about rigorous maintenance Jane.

Lu: I think the title itself, "Skill-V: Verifiable Self-Evolving Skill Library for Interactive Agents," tells you exactly what they're proposing: a library that is verifiable and evolves on its own based on feedback from the world Lu. It’s not just a static list of instructions.

Meng: For practical implementation, the focus seems to be on that contract structure—versioned, falsifiable contracts linking intent to behavior—which gives you that testable element Meng. That structure is what makes it verifiable in an engineering sense.

Lalam: I see this impacting how we think about large models too; instead of just being vast knowledge bases, we could be structured libraries where the links between concepts are constantly being validated against real-world actions Lalam. It could improve culture by making the learning process transparent and provable.

Tom: So, to sum up what this paper is suggesting for the future, Skill-V proposes a method where agents learn not just what to know, but how to manage that knowledge—testing it constantly against the environment to keep it sharp Tom. It’s a principled way to build agents that don't just get bigger, but actually get better at doing things reliably.

Jane: It gives us a framework where agent improvement isn't just about adding new skills from failures, but also about making existing skills more selective and correctable based on how they perform in different situations Jane. That continuous refinement is what separates this approach from older methods that only focus on growth Jane.

Conclusion: Tom: So we’ve seen how Skill-V uses those verifiable contracts and task failures to make agents learn in a really structured way, but let's talk about what that title actually means for us Tom.

Jane: It’s about moving past just letting an agent collect knowledge and instead focusing on making sure the knowledge it has is actively maintained and proven correct against the world Jane.

Lu: The authors are proposing a self-evolving skill library where every skill is basically a contract—a set of rules for what to do, how to do it, and what gets punished if you mess up Lu.

Meng: From an engineering standpoint, this means the AI isn't just getting bigger; it’s getting smarter about which old skills to keep and which new ones to add based on real performance data Meng.

Lalam: I see this as a big deal for culture because it suggests that learning isn't just about absorbing facts; it’s about building a system that constantly checks its own assumptions against reality Lalam.

Tom: Exactly. It’s less like training a student and more like giving them a verifiable curriculum they have to constantly update based on test scores Tom.

Jane: And the authors show that this approach actually leads to state-of-the-art results, showing that this way of evolving is more robust than just letting an agent just keep adding stuff whenever it finds a mistake Jane.

Lu: They highlight how combining skill acquisition with both adding new skills and revising old ones based on disagreements between the contracts and what actually happened gives you that better performance Lu.

Meng: The practical implication for building these things is that you need strong metrics to decide when a revision is safe—you have to make sure you don't break something good just because it failed once Meng.

Lalam: It really shifts the focus from pure growth toward continuous, principled maintenance, which is a much more reliable way for agents to operate in complex environments Lalam.

Tom: That idea of continuous maintenance instead of just unrestrained expansion seems like the big shift here. So what does this actually mean for how we design these next generation agents?

Jie Ma, Zhipeng Qian, Yufei Ma, Zihan Liang, Jiayi Jiayi Ji, Qingpeng Cai, Ben Chen, Peng Jiang

Xiamen University Technology Company

cs.CV

Submitted: 2026-10-08

Updated: 2026-10-08

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 92/100

The gist: The gist The introduction states that reliable skill evolution requires more than accumulating experience; it must learn which knowledge to retain, when to revise it, and when it should be

Key concepts

Skill-V Architecture
This is a verifiable library where each skill is a versioned contract. A skill is defined by its intent (what it aims to do), its rules (how it must behave), and observable criteria. This structure allows the system to formally represent and track what each skill knows and how it should be applied.
Outcome-Grounded Evolution
Skill evolution happens through two signals from the environment. First, task failures prompt the addition of new skills to fill strategy gaps. Second, when there's a mismatch between a skill's contract and what actually happens during a task, it triggers targeted revisions to the skill boundaries.
Revision and Validation
Before any change is made to an existing skill, it must pass strict validation checks. A proposed revision is only accepted if it preserves the original meaning of the skill and doesn't make things worse on previously observed evidence, preventing skills from degrading.

Terminology

Summary

The gist The introduction states that reliable skill evolution requires more than accumulating experience; it must learn which knowledge to retain, when to revise it, and when it should be applied<ref:2610.11781#pg4>.

Skill-V Architecture

Skill-V introduces a verifiable self-evolving skill library where skills are represented as versioned, falsifiable contracts linking semantic intent to observable behavioral criteria<ref:2610.11781#pg6>. Each skill is formally represented as si = (ϕi, pi, ρi, νi), where ϕi specifies the semantic intent and intended applicability of the skill<ref:2610.11781#pg5>. The executable rubric ρi captures observable implications of the skill, including required steps and forbidden behaviors<ref:2610.11781#pg6>. This structure allows Skill-V to revise skills when disagreements between contract evaluations and task outcomes guide revisions to existing skill boundaries<ref:2610.11781#pg4>.

Outcome-Grounded Evolution

Skill-V drives library evolution through two complementary signals derived from environment outcomes<ref:2610.11781#pg4>. First, task failures motivate the addition of new skills, which addresses missing strategies<ref:2610.11781#pg6>. Second, disagreements between contract rubrics and actual outcomes trigger targeted boundary revisions, identifying existing contracts whose boundaries require revision<ref:2610.11781#pg4>. Skill-V aggregates applicable criteria to produce an overall rubric judgment q(τ) and a final contract-level judgment yˆrub(τ) based on a threshold η = 0.7<ref:2610.11781#pg6>.

Revision and Validation

Candidate revisions are committed only when they preserve the skill’s semantic identity and do not regress on previously observed evidence through metrics like Disc, FNR, and BAcc<ref:2610.11781#pg6>. Specifically, a candidate ρ'i is considered evidence-preserving only if it yields a lower or equal Disc and FNR, along with an equal or higher BAcc on the evidence set Ei for the observed evidence set Ei<ref:2610.11781#pg6>. This rigorous constraint allows Skill-V to refine a falsified boundary without sacrificing behavior that remains supported by experience and prevents catastrophic forgetting<ref:2610.11781#pg10>.

Applicability-Aware Use

The library separates semantic relevance from task applicability by first retrieving semantically related candidates and then filtering skills that are confidently inapplicable to the current task<ref:2610.11781#pg4>. Skill-V uses an applicability-aware filter to exclude candidates judged confidently inapplicable to the current task based on a confidence threshold δ = 0.8<ref:2610.11781#pg4>. This process ensures that the selected skills guide the policy while closing the loop between using stored knowledge and testing whether that knowledge should be retained, expanded, or revised<ref:2610.11781#pg4>.

Empirical Results

Skill-V achieves state-of-the-art performance on ALFWorld with a 95.3% success rate and on WebShop with an 85.9% success rate, outperforming existing memory-augmented agents<ref:2610.11781#pg5>. The ablation studies confirm that full Skill-V combines skill acquisition with both directions of contract correction to achieve superior performance compared to methods relying solely on acquisition or failure-only revision<ref:2610.11781#pg9>. Furthermore, the framework demonstrates that verifiable evolution yields state-of-the-art capabilities while maintaining a more compact library than growth-oriented baselines<ref:2610.11781#pg5>.

Conclusion

Skill-V shifts agent learning from pure knowledge accumulation to rigorous knowledge maintenance by using task failures to drive skill addition and leveraging contract-environment disagreements to trigger targeted boundary revisions<ref:2610.11781#pg6>. This approach provides a principled foundation for reliable and autonomously improving agents by prioritizing continuous maintenance over unrestrained expansion<ref:2610.11781#pg10>.

Improvements for AI systems

  1. Bold Header: Verifiable Skill Contracts for Falsifiability

Implement skills as versioned, falsifiable contracts that link semantic intent to observable behavioral criteria. This allows a retrieved skill may be semantically relevant yet inapplicable under the current task conditions and enables systematic testing of existing knowledge.

  1. Bold Header: Outcome-Grounded Boundary Revision

Use disagreements between contract evaluations and task outcomes guide revisions to existing skill boundaries. This mechanism ensures that revisions are driven by empirical evidence, such as when a rubric pass followed by task failure may indicate an overly permissive or incomplete executable specification.

  1. Bold Header: Applicability-Aware Skill Use

Employ a process where skills are first ranked by semantic similarity and then filtered using an LLM judge to exclude candidates judged confidently inapplicable to the current task. This ensures that the agent removes a candidate only when it is judged not applicable with confidence above a threshold δ.

  1. Bold Header: Evidence-Preserving Revision Validation

Commit revisions only if they satisfy rigorous criteria, specifically requiring them to be non-regressive on observed evidence, meaning they must yield lower or equal Disc and FNR, along with an equal or higher BAcc on the evidence set Ei.

  1. Bold Header: Bidirectional Boundary Correction

The system simultaneously utilizes two signals for evolution: task failures expose missing strategies and drive the addition of new skills and rubric–outcome disagreements identify existing contracts whose boundaries require revision, ensuring that reliable skill evolution requires more than adding new knowledge.

  1. Bold Header: Compact, Reliable Library Management

The architecture enables a library size significantly smaller than growth-oriented baselines while maintaining high performance, as Skill-V demonstrates a more compact skill library than growth-oriented baselines by prioritizing verifiable maintenance over unrestrained growth.

Abstract

Interactive agents can turn experience into reusable skills, yet existing self-evolving skill libraries primarily improve by accumulating new knowledge. Failures may lead to new skills, while previously stored skills are less often revisited as new evidence arrives. However, growth alone does not ensure reliability, as a retrieved skill may be inapplicable under the current task conditions, and an existing skill may encode a mis-specified operational boundary. Reliable skill evolution therefore requires not only adding knowledge, but also testing and revising what is already stored. We introduce Skill-V, a verifiable self-evolving skill library. To make stored knowledge testable, we propose representing skills as versioned, falsifiable contracts that link semantic intent to observable behavioral criteria. We use environment outcomes to drive library evolution. Specifically, task failures motivate skill addition, while disagreements between contract evaluations and task outcomes guide revisions to existing skill boundaries. To validate these revisions, we require them to preserve protected semantic constraints and satisfy non-regression criteria for rubric-outcome metrics on historical replay evidence. Finally, we employ an applicability-aware filter to exclude candidates judged confidently inapplicable to the current task. Across ALFWorld and WebShop, Skill-V achieves success rates of 95.3% and 85.9%, respectively, while maintaining a more compact skill library than growth-oriented baselines. Applicability-aware filtering reduces incorrect skill invocations, and outcome-grounded revisions correct mis-specified skill boundaries without degrading performance on previously observed evidence. These results show that reliable skill evolution requires more than accumulating experience: the library must learn which knowledge to retain, when to revise it, and when it should be applied.

Sources

Related papers