V-ICAL Bench: Evaluating Video In-Context Learning for Multimodal Agents in Interactive Environments

arXiv:2609.15683 · cs.CV · Submitted 2026-09-14 · Read on arXiv

cs.CV

Submitted: 2026-09-14

Updated: 2026-10-09

Comments: Code: https://github.com/VisionXLab/V-ICAL Dataset: https://huggingface.co/datasets/VisionXLab/V-ICAL Homepage: https://visionxlab.github.io/V-ICAL-Homepage

Code: https://github.com/VisionXLab/V-ICAL

Project page: https://visionxlab.github.io/V-ICAL-Homepage

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• In-Context Learning — 1× in this paper · explained in 7 episodes

Sources

• Qwen3-VL Technical Report
• Qwen2.5-VL Technical Report
• Demo-ICL: In-Context Learning for Procedural Video Knowledge Acquisition
• Video-MME-v2: Towards the Next Stage in Benchmarks for Comprehensive Video Understanding
• Benchmarking the Spectrum of Agent Capabilities
• Many-Shot In-Context Learning in Multimodal Foundation Models
• Learning Adaptive and Temporally Causal Video Tokenization in a 1D Latent Space
• MM-WebAgent: A Hierarchical Multimodal Web Agent for Webpage Generation
• RISE-Video: Can Video Generators Decode Implicit World Rules?
• Moment-Video: Diagnosing Temporal Fidelity of Video MLLMs on Momentary Visual Events
• GameWorld: Towards Standardized and Verifiable Evaluation of Multimodal Game Agents
• LMAct: A Benchmark for In-Context Imitation Learning with Long Multimodal Demonstrations
• Seed2.0 Model Card: Towards Intelligence Frontier for Real-World Complexity
• Gemma 4 Technical Report
• Kimi K3: Open Frontier Intelligence
• GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learning
• Atari-GPT: Benchmarking Multimodal Large Language Models as Low-Level Policies in Atari Games
• MiMo-VL Technical Report
• DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
• Qwen3-Omni Technical Report

Related papers

• Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
• AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
• Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
• MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
• TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
• A Survey on Efficient Vision-Language-Action Models