VisualCoding: Learning Executable Skills for Reliable Visual Reasoning
cs.CV
Submitted: 2026-10-09
Updated: 2026-10-09
Code: https://github.com/jiaq-liu/VisualCoding
License: http://creativecommons.org/licenses/by/4.0/
The gist: Learning reusable skills can improve visual reasoning agents, yet applying a skill still requires the agent to implement or invoke a procedure and produce the final answer.
Abstract
Learning reusable skills can improve visual reasoning agents, yet applying a skill still requires the agent to implement or invoke a procedure and produce the final answer. Errors and hallucinations at these steps can prevent acquired experience from translating into reliable single-run performance. We propose VisualCoding, which combines execution-guided acquisition of reusable visual programs with direct adoption of their answers on applicable inputs. Execution-Guided Program Acquisition composes visual primitives and additional code into family-level solvers, then iteratively tests and revises them on labeled training instances. Scope-Aware Direct Execution runs a retained solver and returns its valid in-scope answer. We evaluate VisualCoding on four models across four benchmarks spanning 69 task families. VisualCoding exceeds the strongest evaluated baseline in mean accuracy by 4.45 percentage points with Gemini-3.1-Pro and by 14.78 points when reusing the frozen Pro-acquired artifacts with Flash-Lite. Ablations support execution-guided acquisition and show that direct execution outperforms textual or callable reuse of the same programs. Code is available at https://github.com/jiaq-liu/VisualCoding.
Sources
- GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning
- Selective Classification for Deep Neural Networks
- Visual Programming: Compositional visual reasoning without training
- DeepEyesV2: Toward Agentic Multimodal Model
- Don't Show Pixels, Show Cues: Unlocking Visual Tool Reasoning in Language Models via Perception Programs
- XSkill: Continual Learning from Experience and Skills in Multimodal Agents
- SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Tasks
- SkillGen: Verified Inference-Time Agent Skill Synthesis
- Trace2Skill: Distill Trajectory-Local Lessons into Transferable Agent Skills
- ViperGPT: Visual Inference via Python Execution for Reasoning
- Dynamic Cheatsheet: Test-Time Learning with Adaptive Memory
- Voyager: An Open-Ended Embodied Agent with Large Language Models
- Pixel Reasoner: Incentivizing Pixel-Space Reasoning with Curiosity-Driven Reinforcement Learning
- Agent Workflow Memory
- Transductive Visual Programming: Evolving Tool Libraries from Experience for Spatial Reasoning
- Self-Evolving Code-with-Image Reasoning
- SkillOpt: Executive Strategy for Self-Evolving Agent Skills
- Meta Context Engineering via Agentic Skill Evolution
- TextGrad: Automatic "Differentiation" via Text
- MMSkills: Towards Multimodal Skills for General Visual Agents
Related papers
- Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
- AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
- Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
- MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
- TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
- A Survey on Efficient Vision-Language-Action Models