Game2Real-Bench: Do Toy Games Reveal and Transfer to Real-World Visual Reasoning?
cs.CV
Submitted: 2026-10-13
Updated: 2026-10-13
Comments: 32 pages, 8 figures, 11 tables
Code: https://github.com/QingAnLiu/Game2Real-Bench
License: http://creativecommons.org/licenses/by/4.0/
The gist: Visual reasoning is an important capability for vision-language models (VLMs) to understand visual evidence, spatial structure, constraints, and state changes.
Abstract
Visual reasoning is an important capability for vision-language models (VLMs) to understand visual evidence, spatial structure, constraints, and state changes. Existing evaluations mainly rely on real-world data or procedurally generated tasks. Real-world data preserve natural complexity, while procedural games provide controllable generation and verifiable solutions. However, it remains unclear whether abilities measured in toy games correspond to real-world visual reasoning and whether game examples can support transfer. We introduce Game2Real-Bench, a benchmark that studies these connections across four capability groups, including fine-grained perception, spatial reasoning, constraint reasoning, and sequential planning. We construct 1,000 procedurally generated game samples with executable verification and curate 400 real-world samples aligned by shared reasoning requirements rather than visual similarity. We evaluate 25 representative VLMs and observe strong game-to-real consistency, with Pearson and Spearman correlations of 0.935. We further study one-shot game-to-real transfer on four models. Constraint reasoning improves for several models, while fine-grained perception declines with game demonstrations. We also construct Game2Real-Dataset with nearly 24K verified game samples across 18 tasks and 65 QA families. Our benchmark, dataset, and code are available at https://github.com/QingAnLiu/Game2Real-Bench.
Sources
- Qwen3-VL Technical Report
- BabyVision: Visual Reasoning Beyond Language
- EVEv2: Improved Baselines for Encoder-Free Vision-Language Models
- From Pixels to Words -- Towards Native One-Vision Models at Scale
- SenseNova-U1: Unifying Multimodal Understanding and Generation with NEO-unify Architecture
- GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learning
- STEP3-VL-10B Technical Report
- VisualTrans: A Benchmark for Real-World Visual Transformation Reasoning
- Ovis2.5 Technical Report
- SPair-71k: A Large-scale Benchmark for Semantic Correspondence
- Gemma 4 Technical Report
- A Very Big Video Reasoning Suite
- InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency
- VBVR-Pro: A Scalable and Verifiable Suite for Native Visual Reasoning
- Thinking in Structures: Evaluating Spatial Intelligence in Constraint-Governed Spaces
- MiMo-VL Technical Report
- VideoGameBench: Can Vision-Language Models complete popular video games?
Related papers
- Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
- AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
- Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
- MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
- TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
- A Survey on Efficient Vision-Language-Action Models