VIEWCRAFT: Evolving What Multimodal Agents See and How They Read It
cs.CV
Submitted: 2026-10-11
Updated: 2026-10-11
License: http://creativecommons.org/licenses/by/4.0/
The gist: Reusable skills improve multimodal agents without updating model weights, but their effectiveness depends on both the evidence presented and the procedure used to read it.
Abstract
Reusable skills improve multimodal agents without updating model weights, but their effectiveness depends on both the evidence presented and the procedure used to read it. We introduce ViewCraft, which jointly evolves an executable evidence interface and its reading procedure. The interface selects, transforms, and organizes observations; the reader interprets the resulting views. Failure traces guide revisions, replay and paired evaluation retain useful changes, and interface projection recovers visual edits from eligible rejected joint proposals. Across eight benchmarks, ViewCraft improves over direct execution by an average of 6.1 points in task score, with gains on every benchmark. The advantage persists at a matched input-token budget. On all five multi-source tasks, the learned interface uses 8%-15% fewer input tokens than Initial skill. The learned skills also transfer across executors and outperform direct execution in every evaluated transfer, supporting evidence construction and reading as a coupled, reusable skill.
Sources
- Dynamo: Dynamic Skill-Tool Evolution for Vision-Language Agents
- SkillOpt: Executive Strategy for Self-Evolving Agent Skills
- Agent Skills Should Go Beyond Text: The Case for Visual Skills
- MMSkills: Towards Multimodal Skills for General Visual Agents
- XSkill: Continual Learning from Experience and Skills in Multimodal Agents
- EpiCon: Collective Agent Learning through Co-Evolving Multimodal Memory
- Voyager: An Open-Ended Embodied Agent with Large Language Models
- Set-of-Mark Prompting Unleashes Extraordinary Visual Grounding in GPT-4V
- Visual Sketchpad: Sketching as a Visual Chain of Thought for Multimodal Language Models
- Frame-Voyager: Learning to Query Frames for Video Large Language Models
- MementoGUI: Learning Agentic Multimodal Memory Control for Long-Horizon GUI Agents
- WikiSkill: Compiling Agent Experience into Persistent Knowledge for Skill Evolution
- MMCOMPOSITION: Revisiting the Compositionality of Pre-trained Vision-Language Models
- Video-Holmes: Can MLLM Think Like Holmes for Complex Video Reasoning?
- MC-LLaVA: Multi-Concept Personalized Vision-Language Model
- UniCTokens: Boosting Personalized Understanding and Generation via Unified Concept Tokens
- Small-Large Collaboration: Training-efficient Concept Personalization for Large VLM using a Meta Personalized Small VLM
- Jarvis: Towards Personalized AI Assistant via Personal KV-Cache Retrieval
- M2A: Multimodal Memory Agent with Dual-Layer Hybrid Memory for Long-Term Personalized Interactions
- PEARL: Personalized Streaming Video Understanding Model
Related papers
- Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
- AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
- Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
- MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
- TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
- A Survey on Efficient Vision-Language-Action Models