Interactive Navigation Beyond Reachability: Benchmark and Agentic Framework
cs.RO, cs.AI
Submitted: 2026-10-12
Updated: 2026-10-12
Code: https://github.com/ZGCA-HMI-Lab/MultINavhttps:
License: http://creativecommons.org/licenses/by-nc-sa/4.0/
The gist: Existing embodied navigation tasks commonly assume that traversability and object states remain fixed during an episode.
Terminology
Abstract
Existing embodied navigation tasks commonly assume that traversability and object states remain fixed during an episode. Yet reaching a goal may require opening a door to access a room and then opening a container to reveal a hidden target. We formulate interactive navigation beyond reachability, unifying channel-induced changes in spatial connectivity, container-induced changes in target visibility, and their sequential dependencies. We introduce MultINav-Bench, a 3K-episode ObjectGoal benchmark in MolmoSpaces spanning channel, container, and mixed tasks, with interaction-necessity annotations covering required and unnecessary episodes. Its data generation pipeline automates task construction across multiple task types. The evaluation protocol separately measures navigation success, interaction effectiveness, and cost. We further propose MultINav-Agent, a three-stage agentic framework built on a persistent, state-conditioned interaction graph. Semantic interactive perception grounds entities and interaction states; a vision-language model selects task-relevant actions based on expected changes in connectivity and target observability; and post-interaction verification updates the graph to guide subsequent navigation, exploration, or interaction. Experiments on MultINav-Bench show that MultINav-Agent consistently outperforms adapted baselines across channel, container, and mixed tasks.
Sources
- On Evaluation of Embodied Navigation Agents
- AI2-THOR: An Interactive 3D Environment for Visual AI
- MolmoSpaces: A Large-Scale Open Ecosystem for Robot Navigation and Manipulation
- ObjectNav Revisited: On Evaluation of Embodied Agents Navigating to Objects
- Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environments
- CoINS: Counterfactual Interactive Navigation via Skill-Aware VLM
- BEHAVIOR-1K: A Human-Centered, Embodied AI Benchmark with 1,000 Everyday Activities and Realistic Simulation
- Interactive Navigation with Adaptive Non-prehensile Mobile Manipulation
- UniDoorManip: Learning Universal Door Manipulation Policy Over Large-scale and Diverse Door Manipulation Environments
- Hydra: A Real-time Spatial Perception System for 3D Scene Graph Construction and Optimization
- ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planning
- FunGraph: Functionality Aware 3D Scene Graphs for Language-Prompted Scene Interaction
- MomaGraph: State-Aware Unified Scene Graphs with Vision-Language Model for Embodied Task Planning
- Pandora: Articulated 3D Scene Graphs from Egocentric Vision
- Articulated 3D Scene Graphs for Open-World Mobile Manipulation
- The One RING: a Robotic Indoor Navigation Generalist
- Ground Slow, Move Fast: A Dual-System Foundation Model for Generalizable Vision-and-Language Navigation
Related papers
- FMT x: An Efficient and Asymptotically Optimal Extension of the Fast Marching Tree for Dynamic Replanning
- MPCFormer: A physics-informed data-driven approach for explainable socially-aware autonomous driving
- RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies
- ARC: A Reasoning Recipe for Robot Foundation Models
- HRDexDB: A 4D Dexterous Grasping Dataset Across Human and Multiple Robot Embodiments
- APT: Action Expert Pretraining Improves Instruction Generalization of Vision-Language-Action Policies