Holi-Spatial 360: Generative Reconstruction from Sparse Panoramas for Spatial Rendering and Intelligence
cs.CV
Submitted: 2026-10-14
Updated: 2026-10-14
License: http://creativecommons.org/licenses/by/4.0/
The gist: Spatial intelligence requires scalable 3D environments that are both photorealistic and semantically grounded, yet existing indoor datasets are dominated by room-scale captures and costly LiDAR
Abstract
Spatial intelligence requires scalable 3D environments that are both photorealistic and semantically grounded, yet existing indoor datasets are dominated by room-scale captures and costly LiDAR acquisition. Online real-estate platforms offer abundant panoramas of diverse multi-room homes, but sparse viewpoints and texture-poor regions make reconstruction severely under-constrained. We introduce Holi-Spatial 360, a generative reconstruction and data curation framework for converting these panoramas into annotated multi-room 3D environments. Because reconstruction quality bounds subsequent generation, our Correspondence-Guided Geometric Optimization converts feed-forward reconstruction priors into reliable cross-view constraints and builds a coherent 3D Gaussian Splatting representation as 3D spatial memory. Adapted single-image and video priors then restore appearance and complete continuous trajectories while preserving observed content. The reconstructed scenes support arbitrary RGB-D rendering and an automated curation pipeline that combines end-to-end 3D grounding with VLM-based verification. Holi360 contains over 1K multi-room 3DGS scenes, more than 100K instance annotations, and 2M spatial question-answer pairs. We also introduce the Holi360 benchmark to evaluate the spatial understanding of VLMs in real-world, multi-room indoor environments. Experiments demonstrate superior reconstruction and curation quality under sparse panoramic inputs, while fine-tuning VLMs on Holi360 substantially improves spatial reasoning capabilities.
Sources
- Qwen3-VL Technical Report
- Qwen2.5-VL Technical Report
- Scaling Spatial Intelligence with Multimodal Foundation Models
- PGSR: Planar-based Gaussian Splatting for Efficient and High-Fidelity Surface Reconstruction
- GaussianPro: 3D Gaussian Splatting with Progressive Propagation
- Boxer: Robust Lifting of Open-World 2D Bounding Boxes to 3D
- Holi-Spatial: Evolving Video Streams into Holistic 3D Spatial Intelligence
- PanoWorld: A Generative Spatial World Model for Consistent Whole-House Panorama Synthesis
- Depth Anything 3: Recovering the Visual Space from Any Views
- 3DGS-Enhancer: Enhancing Unbounded 3D Gaussian Splatting with View-consistent 2D Diffusion Priors
- SpatialLM: Training Large Language Models for Structured Indoor Modeling
- One4D: Unified 4D Generation and Reconstruction via Decoupled LoRA Control
- Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AI
- Gemini: A Family of Highly Capable Multimodal Models
- ArtifactWorld: Scaling 3D Gaussian Splatting Artifact Restoration via Video Generation Models
- Habitat-Matterport 3D Semantics Dataset
- Visual Spatial Tuning
- NeoVerse: Enhancing 4D World Model with in-the-wild Monocular Videos
- GSFixer: Improving 3D Gaussian Splatting with Reference-Guided Video Diffusion Priors
- LLaVA-3D: A Simple yet Effective Pathway to Empowering LMMs with 3D-awareness
Related papers
- Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
- AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
- Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
- MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
- TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
- A Survey on Efficient Vision-Language-Action Models