SPW-Nav: A Streaming Panoramic World Model for Language-Guided Navigation
cs.CV
Submitted: 2026-10-06
Updated: 2026-10-06
Comments: Project page: https://alaya-lab.github.io/SPW-Nav
Project page: https://alaya-lab.github.io/SPW-Nav
License: http://creativecommons.org/licenses/by/4.0/
The gist: Language-guided panoramic video generation benefits various downstream applications, such as interactive 3D scene exploration, virtual reality experiences, and embodied agent training.
Terminology
Abstract
Language-guided panoramic video generation benefits various downstream applications, such as interactive 3D scene exploration, virtual reality experiences, and embodied agent training. Existing panoramic generators follow predefined trajectories, and interactive world models act through low-level actions in perspective views. We propose SPW-Nav, a streaming panoramic world model that understands movement instructions and streams one minute of 2K 360-degree video in real time from a single panorama. SPW-Nav interprets each instruction in the previously generated panorama as camera motion. Spherical rotation decoupling applies rotation exactly on the sphere, pose-aligned conditioning keeps translation inputs bounded over long streams, and a multi-term memory with a few-step generator continues the scene as instructions change. We also build SPW-NavSet, panoramic videos with camera trajectories and verified instructions. Driven by language, SPW-Nav outperforms prior panoramic generators in camera-following accuracy and video quality, and supports on-the-fly instruction switching.
Sources
- Qwen3-VL Technical Report
- Video Generation Models Are Inherent Lighting Estimators
- World Models
- Matrix-game 2.0: An open-source, real-time, and streaming interactive world model
- Video Diffusion Models
- Sol-Attn: Accelerating Video Generation Inference via On-the-Fly Attention Sparsification
- PanoWorld: Real-World Panoramic Generation
- CubeComposer: Spatio-Temporal Autoregressive 4K 360{\deg} Video Generation from Perspective Video
- TARS: Timestep-Aware Data Scaling for 3D-Free Video Re-Shooting
- Yume: An Interactive World Generation Model
- Make-A-Video: Text-to-Video Generation without Text-Video Data
- The Replica Dataset: A Digital Replica of Indoor Spaces
- Wan: Open and Advanced Large-Scale Video Generative Models
- Matrix-Game 3.0: Real-Time and Streaming Interactive World Model with Long-Horizon Memory
- Helios: Real Real-Time Long Video Generation Model
- Matrix-Game: Interactive World Foundation Model
- PhyWorld: Physics-Faithful World Model for Video Generation
- CamI2V: Camera-Controlled Image-to-Video Diffusion Model
Related papers
- Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
- AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
- Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
- MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
- TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
- A Survey on Efficient Vision-Language-Action Models