MemAct: Streaming Spatial Understanding with Memory and Active Exploration
cs.CV
Submitted: 2026-10-13
Updated: 2026-10-13
Project page: https://vfishc.github.io/memact/ABSTRACT
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- ScanQA: 3D Question Answering for Spatial Scene Understanding
- ARKitScenes: A Diverse Real-World Dataset For 3D Indoor Scene Understanding Using Mobile RGB-D Data
- SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities
- LiveCC: Learning Video LLM with Streaming Speech Transcription at Scale
- SpatialRGPT: Grounded Spatial Reasoning in Vision Language Models
- ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes
- S-Agent: Spatial Tool-Use Elicits Reasoning for Spatial Intelligence
- Embodied Question Answering
- VLM-3R: Vision-Language Models Augmented with Instruction-Aligned 3D Reconstruction
- BLINK: Multimodal Large Language Models Can See but Not Perceive
- Enter the Mind Palace: Reasoning and Planning for Long-term Active Embodied Question Answering
- IQA: Visual Question Answering in Interactive Environments
- ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planning
- MA-LMM: Memory-Augmented Large Multimodal Model for Long-Term Video Understanding
- ESI-Bench: Towards Embodied Spatial Intelligence that Closes the Perception-Action Loop
- LoRA: Low-Rank Adaptation of Large Language Models
- Visual Language Maps for Robot Navigation
- Online Video Understanding: OVBench and VideoChat-Online
- ConceptFusion: Open-set Multimodal 3D Mapping
- ViewSpatial-Bench: Evaluating Multi-perspective Spatial Localization in Vision-Language Models
Related papers
- Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
- AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
- Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
- MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
- TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
- A Survey on Efficient Vision-Language-Action Models