ActiveLang: Active Open-Vocabulary 3D Mapping with Semantic-Uncertainty-Guided Exploration

arXiv:2610.09518 · cs.CV, cs.RO · Submitted 2026-10-07 · Read on arXiv

cs.CV, cs.RO

Submitted: 2026-10-07

Updated: 2026-10-07

Code: https://github.com/facebookresearch/Replica-Dataset

License: http://creativecommons.org/licenses/by/4.0/

The gist: As robots increasingly assist humans with diverse tasks, they need both geometric and semantic understanding of their surroundings.

Terminology

Abstract

As robots increasingly assist humans with diverse tasks, they need both geometric and semantic understanding of their surroundings. Moreover, robots often operate in unfamiliar environments and take on new tasks without knowing the relevant concepts ahead of time. This motivates language-annotated 3D maps that support open-vocabulary scene understanding and human-robot interaction. We introduce ActiveLang, an autonomous system for active open-vocabulary 3D mapping with semantic-uncertainty-guided exploration. ActiveLang performs online language-feature adaptation on a compact dual-Gaussian representation to jointly reconstruct scene geometry, appearance, and open-vocabulary semantics with modest memory overhead. Its planner efficiently selects informative viewpoints, enabling effective mapping with fewer observations and lower computational cost. Experiments on Replica and ScanNet++ demonstrate substantial improvements in 2D and 3D open-vocabulary segmentation over both online and offline baselines, highlighting that actively exploring scenes builds language-annotated 3D maps more efficiently.

Sources

Related papers