EgoProbe: Benchmarking Long-Range and Exact Memory in Streaming Egocentric Videos
cs.CV
Submitted: 2026-10-12
Updated: 2026-10-12
Comments: 22 pages, 9 figures, 8 tables
License: http://creativecommons.org/licenses/by/4.0/
The gist: Egocentric video provides a continuous first-person record of everyday activities, making it a natural foundation for wearable AI assistants.
Terminology
Abstract
Egocentric video provides a continuous first-person record of everyday activities, making it a natural foundation for wearable AI assistants. Unlike offline video understanding, streaming systems must process observations as they arrive, while queries may refer to experiences from the distant past. Recent work has begun to explore whether information can be recalled across long temporal gaps. Yet remembering how far is only part of the challenge: recurring objects, actions, and locations also require models to remember exactly which detail belongs to which event. We introduce EgoProbe, a benchmark that evaluates streaming egocentric memory along these two complementary dimensions: long-range retention, i.e., how far, and event-specific recall under interference, i.e., how exact. Built on long-form egocentric videos, EgoProbe uses an event-grounded data engine to extend recall intervals, identify related events, and construct event-grounded distractors, followed by human verification. It contains two subsets: EgoProbe-Base, which evaluates long-range recall without intervening related events, and EgoProbe-Echo, which challenges models with similar intervening events and event-grounded distractors. EgoProbe-Echo adopts coarse-to-fine multi-turn probing: the first turn evaluates coarse-grained event recall, while the second tests whether the model can retrieve the target-specific detail associated with the designated event. Extensive evaluation across diverse models reveals substantial limitations in long-range recall, with further challenges in recovering target-event details among related experiences. Even when models correctly answer coarse-grained questions, they often fail to recall the specific details of the designated event. EgoProbe exposes this underexplored failure mode and provides a testbed for advancing long-range and exact memory in streaming egocentric video understanding.
Sources
- GPT-4 Technical Report
- EM^2Mem: Event-Centric Multimodal Memory for Large Language Models
- ObjectStream: Latent Objects as Memory Anchors for Streaming Video Understanding
- EGOSTREAM: A Diagnostic Benchmark for Streaming Episodic Memory in Egocentric Vision
- LLaVA-OneVision: Easy Visual Task Transfer
- MEMO: Multi-Level Entity-Aware Memory for Streaming Video Understanding
- OASIS: On-Demand Hierarchical Event Memory for Streaming Video Reasoning
- Evo-Depth: A Lightweight Depth-Enhanced Vision-Language-Action Model
- LA4VLA: Learning to Act without Seeing via Language-Action Pretraining
- Exo2EgoSyn: Unlocking Foundation Video Generation Models for Exocentric-to-Egocentric Video Synthesis
- LongVU: Spatiotemporal Adaptive Compression for Long Video-Language Understanding
- A Simple Baseline for Streaming Video Understanding
- RIVER: A Real-Time Interaction Benchmark for Video LLMs
- Gemini: A Family of Highly Capable Multimodal Models
- Qwen3 Technical Report
- Ego-R1: Chain-of-Tool-Thought for Ultra-Long Egocentric Video Reasoning
- OFlow: Injecting Object-Aware Temporal Flow Matching for Robust Robotic Manipulation
- Afford-VLA: Action-Aligned Visual Planning via Internalized Affordance
- InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency
- S-EMBER: A Large-Scale Benchmark for Streaming Egocentric Memory Retrieval
Related papers
- Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
- AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
- Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
- MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
- TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
- A Survey on Efficient Vision-Language-Action Models