ASPIRE: Agentic /Skills Discovery for Robotics

Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.

Rosa: Hi, I'm Rosa.

Dev: And I'm Dev, here with Taro.

Rosa: We're looking at the paper: "ASPIRE: Agentic /Skills Discovery for Robotics".

Dev: Here's the gist of the paper. Traditional robot programming is notoriously challenging, requiring orchestration of multimodal perception, management of complex physical contact dynamics, and handling diverse environment configurations and execution failures The gist Aspire enables automated discovery of reusable skills that persist across multiple tasks, simulation and real-world settings, and different embodiments How it works Aspire is a continual learning system for robotics that autonomously writes and refines robot control programs in a code-as-policy paradigm while compounding experience into a reusable skill library It operates in an open-ended learning loop consisting of three key components (1) a closed-loop robot execution engine that exposes fine-grained multimodal traces, enabling the agent to autonomously diagnose failures, synthesize repairs, and validate outcomes; (2) a continually expanding skill library that distills validated fixes into reusable, transferable robotic knowledge; and (3) an evolutionary search procedure that generates diverse task sequences and control programs Robot Execution Engine The closed-loop robot execution engine replaces coarse rollout-level feedback with per-primitive execution traces For each primitive call, the trace stores the invoked API, inputs and outputs, return status, and relevant multimodal evidence such as RGB keyframes, overlays, grasp candidates, object poses, and motion-planning results These rich multimodal traces allow the agent to selectively inspect salient primitive logs to progressively localize failures For instance, in a navigation task where candidate navigation goals fall inside the collision-avoidance buffer due to table constraints, the trace localizes the failure to "repeated PLANNING ERRORs" The agent then patches the program with a multi-angle approach routine, and this validated repair is admitted as a reusable Multi-Angle Approach skill Skill Library Aspire maintains a growing skill library that distills validated fixes into reusable, transferable robotic knowledge retrievable as in-context guidance for future tasks This library stores heterogeneous repair knowledge such as localization heuristics, perception prompts, grasping constraints, and debugging workflows Each skill is stored as compact in-context guidance including the failure signature, when-to-apply condition, repair strategy, and a representative code sketch The coordinator audits these findings and promotes only reusable repairs that have passed debug validation into the shared library Evolutionary Search Aspire employs an evolutionary search procedure to generate diverse task sequences and control programs, exploring beyond single-trajectory self-improvement through iterative debugging and parallel refinement In each round, based on the skill library, the coding agent proposes a population of K candidate programs conditioned on the top-performing previous programs and failure traces from previous evaluations The search target is the robot program itself, and candidates are selected through closed-loop execution to explore distinct strategies rather than repeatedly refining the same solution This procedure allows Aspire to scale its performance with experience, as "the more tasks Aspire sees, the larger its skill library grows" Empirical Results and Transfer Aspire demonstrates strong self-improvement across diverse short- and long-horizon robot benchmarks Against prior coding agents, Aspire improves success rate by up to 77 points on LIBERO-Pro under perturbation suites, by up to 72 points on Robosuite contact-rich manipulation tasks, and by up to 32% on long-horizon household tasks Based on the skill library accumulated on LIBERO-90, Aspire transfers zero-shot to LIBEROPro Long with 31% success, substantially outperforming the 4% success rate of prior methods despite their heavy reliance on test-time reasoning and retries Furthermore, skills discovered in simulation provide initial evidence of sim-to-real transfer by reducing real-robot programming effort despite different embodiments and robot APIs Real-robot cross-embodiment skill transfer shows that retrieving simulation-discovered skills as in-context guidance reduces real-world reasoning tokens and enables successful programs where naive debugging without skill transfer fails entirely Limitations Aspire has several important limitations, including not yet being a fully autonomous real-world lifelong learner due to the need for robust success detection and safe reset in the physical world The method also relies on a frozen frontier LLM (Claude Opus 4.6 with a 1M-token context window) to interpret multimodal traces, write program repairs, and propose evolutionary-search candidates Finally, the debug and evolutionary-search loop is compute-intensive, consuming many LLM calls and simulator or robot rollouts per task The gist Aspire enables automated discovery of reusable skills that persist across multiple tasks, simulation and real-world settings, and different embodiments The robot execution engine records per-primitive multimodal traces for perception, planning, and control calls, exposing the trace to the coding agent to focus on evidence around calls implicated by the failure Aspire builds a self-improving continual learning robotic system that autonomously writes and refines robot control programs while accumulating experience into a reusable skill library Across diverse benchmarks, Aspire substantially outperforms existing VLA and coding-agent baselines, demonstrating strong zero-shot transfer to unseen long-horizon tasks How it works The agent pipeline involves a coordinator managing the shared skill library and dispatching actor coding agents to individual tasks in a parallel manner The subagent's role is to diagnose failed robot trials for one task, write a generalizable repair program, validate it according to the protocol, and report concise results The coordinator then promotes generalizable patterns into the shared skill library after each completion Agent Pipeline Skills The agent system prompt instructs Claude Code to read project memory first and adhere to forbidden APIs that access simulator ground truth or internal state It is encouraged to design new helper functions when finding repeating patterns and add them to the relevant skill in.

Title and authors: Rosa: So we're looking at this paper called "ASPIRE: Agentic /Skills Discovery for Robotics." It's by Runyu Lu, Yubo Wu, Ethan Kou, Letian Fu, Wenli Xiao, Ajay Mandlekar, Yinzhen Xu, Guanya Shi one through five <ref:2607.00272#pg1>.

Dev: Yeah it’s a system that tries to tackle the really hard part of robot programming which is handling all the different kinds of perception and physical contact stuff at once.

Rosa: Exactly. The title says Agentic Skill Discovery for Robotics, and it points toward building these skills automatically instead of having people write every single line of code by hand.

Dev: It's about creating this continual learning system that writes and refines robot control programs while adding experience to a reusable skill library, which is the core idea here.

Taro: I wonder how it handles the fact that real-world environments are so messy compared to simulation, since we're talking about these kinds of skills.

Rosa: That's a big question for us because the paper suggests these skills should actually persist across multiple tasks, even when you move from simulation to the real world.

Dev: And they focus on how this system operates in an open-ended learning loop consisting of three main parts, which is pretty neat.

Taro: So we're talking about an execution engine that gives us detailed data, a library for storing those fixes, and then some kind of search procedure to find new ways to do things.

The paper's summary: Rosa: The paper says the core idea is replacing the old way of getting feedback with something much more detailed. They introduce this closed-loop robot execution engine that records per-primitive multimodal traces for every single action a robot takes, like perception or planning.

Dev: That means instead of just getting a success or failure signal at the end of a long sequence, you get data on every little step, including RGB keyframes and motion planning results.

Rosa: That rich multimodal trace information lets the agent autonomously diagnose failures and then synthesize repairs by selectively inspecting those logs to pinpoint exactly where things went wrong.

Taro: So if a navigation plan fails because it runs into a table constraint, the system can look at the trace and figure out that the failure is coming from repeated planning errors in that specific spot.

Dev: Right, and after finding that error, it patches the program with a multi-angle approach routine to fix it, and then this validated repair becomes a reusable skill.

Rosa: Then they have this growing skill library where these validated fixes are distilled into reusable robotic knowledge, which can be retrieved later as in-context guidance for totally new tasks.

Taro: That part about the skill library seems important because it means experience isn't just stuck in one robot's code, but it becomes transferable knowledge.

Dev: And they also have this evolutionary search procedure that generates diverse task sequences and control programs by proposing a population of candidate programs based on what worked before.

The paper's improvements: Rosa: The main improvement they highlight is moving away from fixed, human-engineered pipelines to this autonomous system that writes and refines robot control programs in a code-as-policy paradigm.

Dev: That means the system isn't just following pre-set instructions; it’s actually figuring out the necessary code itself through iterative debugging.

Rosa: They emphasize how the execution engine exposes these per-primitive multimodal traces, which lets the agent focus its inspection on salient logs instead of just looking at everything at once.

Taro: And they show that this iterative debugging process works by letting the agent re-execute and validate its own repairs, which is a key part of making sure the fix actually works before it's saved.

Dev: They build this skill library by distilling validated fixes into compact in-context guidance, storing things like failure signatures and repair strategies.

Rosa: And they stress that this system allows skills discovered in simulation to be adapted as cross-embodiment guidance for real-robot programming, which is a big deal for sim-to-real transfer.

Taro: So it’s not just about learning one specific task well, but about the system accumulating enough diverse skills so it can handle many different things gracefully.

Conclusion: Rosa: To wrap up, ASPIRE is essentially a continual learning system that autonomously writes and refines robot control programs by compounding experience into a transferable skill library across tasks, simulation, and different embodiments.

Dev: It’s about this closed-loop process where the execution engine provides the data to diagnose failures, the skill library stores the fixes so they can be reused as guidance for future tasks, and the evolutionary search generates diverse programs to keep exploring.

Taro: I think what really stands out is how it shows that skills discovered in simulation provide initial evidence of sim-to-real transfer by reducing real-robot programming effort even when the robot has different embodiments or APIs.

Rosa: And they show zero-shot generalization across diverse benchmarks, meaning after learning a set of skills, ASPIRE can perform well on unseen long-horizon tasks with success rates up to thirty-one percent.

Dev: But the authors also admit that it's not a fully autonomous real-world lifelong learner yet because they still need robust success detection and safe resets when things go wrong physically.

Taro: And they also note that this whole loop is quite compute-intensive, consuming a lot of LLM calls and simulator or robot rollouts for each task, which is something to keep in mind.

Rosa: So ASPIRE is a strong system for discovering reusable skills by automating the writing and refining of robot control programs, showing real promise for making robots more adaptable.