Open-o3-Video-v2: Seek, Ground, and Reason with Verifiable Spatio-Temporal Evidence
cs.CV
Submitted: 2026-10-13
Updated: 2026-10-13
Comments: Extension of Open-o3-Video (https://arxiv.org/abs/2510.20579). V2 extends single-pass grounded reasoning to agentic multi-turn evidence seeking and grounding, enabling models to actively acquire and reason over missing visual evidence. We further introduce STGR-v2-CoT-135K and CS-GRPO to improve evidence acquisition and grounded reasoning
Code: https://github.com/modelscope/ms-swift2https:
Project page: https://msalab-pku.github.io/projects/Open-o3-Video-v2
License: http://creativecommons.org/licenses/by-nc-nd/4.0/
The gist: Reliable video reasoning requires models to produce correct answers and expose supporting visual evidence.
Abstract
Reliable video reasoning requires models to produce correct answers and expose supporting visual evidence. Open-o3-Video enables verifiable reasoning by embedding timestamps and object-level bounding boxes into reasoning traces, but is restricted to frames from an initial sparse sampler. We introduce Open-o3-Video-v2, which formulates video reasoning as a unified agentic evidencegrounded process. The model identifies missing information, acquires additional observations, grounds the moments and objects supporting its reasoning, and writes this evidence into the trajectory before continuing or answering. To support this behavior, we construct STGR-v2-CoT135K, including 52K multi-turn agentic grounded reasoning trajectories, and STGR-v2-RL-13K for reinforcement learning. We further propose Clue-Seek GRPO (CS-GRPO), which tracks how tool calls change accumulated evidence, rewards clue coverage and step-wise gains, and penalizes excessive, repetitive, or uninformative retrieval. On V-STaR, Open-o3-Video-v2-4B achieves 40.4% mAM and 56.3% mLGM, outperforming its Qwen3-VL-4B backbone by 8.5% and 12.6%, respectively. It also delivers consistent gains across general video understanding, long-video reasoning, and temporal grounding benchmarks. These results show that active evidence acquisition and verifiable object-level grounding are complementary for reliable video reasoning. The code and models are released at https://msalab-pku.github.io/projects/Open-o3-Video-v2.
Sources
- LLaVA-OneVision-2: Towards Next-Generation Perceptual Intelligence
- Qwen2.5-VL Technical Report
- OmniVideo-100K: A Dataset for Audio-Visual Reasoning through Structured Scripts and Evidence Chains
- SAM 3: Segment Anything with Concepts
- CG-Bench: Clue-grounded Question Answering Benchmark for Long Video Understanding
- Datasets and Recipes for Video Temporal Grounding via Reinforcement Learning
- Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling
- Video-Holmes: Can MLLM Think Like Holmes for Complex Video Reasoning?
- V-STaR: Benchmarking Video-LLMs on Video Spatio-Temporal Reasoning
- Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities
- Reinforcing Video Reasoning with Focused Thinking
- VideoZoomer: Reinforcement-Learned Temporal Focusing for Long Video Reasoning
- Video-of-Thought: Step-by-Step Video Reasoning from Perception to Cognition
- Video-R1: Reinforcing Video Reasoning in MLLMs
- Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis
- FrameMind: Frame-Interleaved Video Reasoning via Reinforcement Learning
- Chain-of-Frames: Advancing Video Understanding in Multimodal LLMs via Frame-Aware Reasoning
- Thinking With Bounding Boxes: Enhancing Spatio-Temporal Video Grounding via Reinforcement Fine-Tuning
- WorldSense: Evaluating Real-world Omnimodal Understanding for Multimodal LLMs
- Video-MMMU: Evaluating Knowledge Acquisition from Multi-Discipline Professional Videos
Related papers
- Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
- AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
- Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
- MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
- TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
- A Survey on Efficient Vision-Language-Action Models