ViSkill: Reinforcing VLM Agents with Evolving Visual-Native Skills
cs.CV, cs.CL
Submitted: 2026-10-08
Updated: 2026-10-08
Comments: Code: https://github.com/ZJU-REAL/ViSkill
Code: https://github.com/ZJU-REAL/ViSkill
License: http://creativecommons.org/licenses/by/4.0/
The gist: Skill-augmented agents improve sample efficiency by distilling successful trajectories into reusable strategies.
Terminology
Abstract
Skill-augmented agents improve sample efficiency by distilling successful trajectories into reusable strategies. Yet most existing approaches remain text-centric, linearizing spatial layouts and action-state correspondences into language that loses critical geometric structure. Recent efforts have begun incorporating visual evidence, but construct and update skills separately from policy optimization, leaving their mutual improvement underexplored. We propose ViSkill, a visual-native skill learning framework that encodes successful interactions as composite visual skill cards directly accessible to VLM agents. Retrieved skills guide both inference and reward shaping, while successful trajectories are distilled back into the library, forming a closed feedback loop in which skill accumulation and policy improvement reinforce each other. An optional cold-start mechanism further accelerates early-stage learning. Evaluated on Sokoban, FrozenLake, and PrimitiveSkill, ViSkill achieves an overall success rate of 0.89, rising to 0.91 with cold-start initialization, outperforming all evaluated proprietary and open-source baselines while converging faster than standard PPO. Our code is available at https://github.com/ZJU-REAL/ViSkill.
Sources
- Qwen2.5-VL Technical Report
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- Skill-CMIB: Multimodal Agent Skill for Consistent Action via Conditional Multimodal Information Bottleneck
- GPT-4o System Card
- XSkill: Continual Learning from Experience and Skills in Multimodal Agents
- Perceive-to-Reason: Decoupling Perception and Reasoning for Fine-Grained Visual Reasoning
- MUSE-Autoskill: Self-Evolving Agents via Skill Creation, Memory, Management, and Evaluation
- SkillLens: Visual Skill Cards for Retrieval-Augmented GUI Action Prediction and On-Policy Distillation
- Visual Agentic Reinforcement Fine-Tuning
- SKILL0: In-Context Agentic Reinforcement Learning for Skill Internalization
- ManiSkill: Generalizable Manipulation Skill Benchmark with Large-Scale Demonstrations
- Trace2Skill: Distill Trajectory-Local Lessons into Transferable Agent Skills
- DINOv2: Learning Robust Visual Features without Supervision
- Proximal Policy Optimization Algorithms
- VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Model
- Skill1: Unified Evolution of Skill-Augmented Agents via Reinforcement Learning
- Voyager: An Open-Ended Embodied Agent with Large Language Models
- AtlasVA: Self-Evolving Visual Skill Memory for Teacher-Free VLM Agents
- RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learning
- Milestone-Guided Policy Learning for Long-Horizon Language Agents
Related papers
- Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
- AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
- Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
- MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
- TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
- A Survey on Efficient Vision-Language-Action Models