Flow-of-Thought: A Framework for Visual Reasoning
cs.CV
Submitted: 2026-10-07
Updated: 2026-10-07
Comments: NeurIPS WiML 2026 version: OpenReview version: https://openreview.net/forum?id=QBcqVOacYO
License: http://creativecommons.org/licenses/by/4.0/
The gist: Mental imagery, ``seeing with the mind's eye'' is an essential aspect of human cognition.
Terminology
Abstract
Mental imagery, ``seeing with the mind's eye'' is an essential aspect of human cognition. Despite rapid progress Large Language Models (LLMs) and Vision Transformers (ViTs) still underperform on tasks requiring spatial understanding. To address this, we introduce Flow-of-Thought (FoT), a framework that integrates the generation of visual sketches as intermediate reasoning steps, mimicking mental imagery in humans. We train coordinate-aware trajectory flow fields on SO(2) group orbits and cumulative shortest paths, then freeze the learned dynamics; same vs. different decisions compare competing generative hypotheses using foreground-weighted reconstruction energy. On locked tests FoT reaches 100.0% accuracy on Tetris and 99.0% on colored shapes. Under frozen transfer, the orbit-trained 2D flow improves over its endpoint-only control on BLINK Multi-view (72.2% vs. 63.9% on 133 public validation pairs), supporting continuous visual traces as an effective and interpretable representation for spatial reasoning in some out-of-distribution settings.
Sources
- On the Measure of Intelligence
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- Self-Supervised Visual Preference Alignment
- Let's Verify Step by Step
- Flow Matching for Generative Modeling
- Chain-of-Sketch: Enabling Global Visual Reasoning
- TopoBench: Benchmarking LLMs on Hard Topological Reasoning
- VSP: Assessing the dual challenges of perception and reasoning in spatial planning tasks for VLMs
- Latent Sketchpad: Sketching Visual Thoughts to Elicit Multimodal Reasoning in MLLMs
Related papers
- Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
- AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
- Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
- MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
- TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
- A Survey on Efficient Vision-Language-Action Models