RAR: Rate-Adaptive Rollout for Time-Consistent Interactive World Models
cs.CV
Submitted: 2026-10-12
Updated: 2026-10-12
Comments: Project page: https://hkust-longgroup.github.io/RAR
Project page: https://hkust-longgroup.github.io/RAR
License: http://creativecommons.org/licenses/by/4.0/
The gist: Video-based interactive world models aim to enable real-time, user-controllable environments.
Terminology
Abstract
Video-based interactive world models aim to enable real-time, user-controllable environments. Yet most existing systems are trained and rolled out at a fixed temporal rate, limiting both runtime-rate adaptation and physical-horizon coverage -- a limitation we term temporal-rate coupling. Specifically, when runtime generation throughput differs from the learned temporal rate, the generated world evolves unintentionally more slowly or quickly. Moreover, using a single temporal rate imposes an inherent trade-off between temporal granularity and physical horizon: higher rates capture finer motion but cover a shorter duration under a fixed context window. To address both limitations, we introduce Rate-Adaptive Rollout (RAR), a framework that enables interactive world models to roll out at controllable temporal rates. At its core, Stride-RoPE aligns latent frames sampled at different rates on a shared physical-time coordinate, enabling consistent multi-rate rollout, unseen-rate generalization, and online rate switching without discarding history. Meanwhile, RAR uses multi-stride video sampling to enable joint learning of fine-grained dynamics and long-horizon evolution. Across action- and camera-controlled environments, RAR achieves temporally consistent multi-rate rollout and online rate switching while preserving high-rate generation quality and extending physical horizons under matched training budgets. RAR also transfers to pretrained world models while preserving perceptual quality.
Sources
- Echo-Infinity: Learning Evolving Memory for Real-Time Infinite Video Generation
- Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets
- SkyReels-V2: Infinite-length Film Generative Model
- VideoCrafter1: Open Diffusion Models for High-Quality Video Generation
- LongRoPE: Extending LLM Context Window Beyond 2 Million Tokens
- DreamX-World 1.0: A General-Purpose Interactive World Model
- Infinite Worlds with Versatile Interactions
- World Models
- LTX-Video: Realtime Video Latent Diffusion
- LTX-2: Efficient Joint Audio-Visual Foundation Model
- Matrix-game 2.0: An open-source, real-time, and streaming interactive world model
- Imagen Video: High Definition Video Generation with Diffusion Models
- GAIA-1: A Generative World Model for Autonomous Driving
- Multiplayer Interactive World Models with Representation Autoencoders
- Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusion
- Sekai: A Video Dataset towards World Exploration
- MultiGen: Level-Design for Editable Multiplayer Worlds in Diffusion Game Engines
- Solaris: Building a Multiplayer Video World Model in Minecraft
Related papers
- Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
- AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
- Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
- MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
- TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
- A Survey on Efficient Vision-Language-Action Models