Can a Frontier VLM Serve as a Robot Policy for Memory-Dependent Manipulation?
cs.RO
Submitted: 2026-10-12
Updated: 2026-10-16
Comments: 13 pages, 6 figures, 5 tables. Project website: https://embodied-astra.github.io
Code: https://github.com/jackbjed/embodied-astra-robomme
Project page: https://embodied-astra.github.io
License: http://creativecommons.org/licenses/by/4.0/
The gist: We use a frozen frontier vision-language model, GPT-6-Astra, directly as the robot policy on RoboMME, a benchmark of 16 memory-dependent manipulation tasks spanning temporal, spatial, object, and
Abstract
We use a frozen frontier vision-language model, GPT-6-Astra, directly as the robot policy on RoboMME, a benchmark of 16 memory-dependent manipulation tasks spanning temporal, spatial, object, and procedural memory. The model keeps the whole episode in context and outputs every end-effector waypoint and gripper command, with no RoboMME-specific training. On the 800 test episodes it succeeds in 92.1%, above the previously reported human score of 90.5%, which was obtained with an oracle skill interface, and far ahead of the best leaderboard entry (60.8%) and of a hierarchical system that pairs the same model with a RoboMME-trained policy (79.1%). It matches or exceeds the human score on 13 of the 16 tasks. Its 63 failures concentrate in two tasks and are mostly premature completions and mistimed or unintended button presses. These results blur the usual division between long-horizon planning, memory and end-effector control, and raise the question of how much of this stack a single general-purpose model can take over in embodied systems. Project website: https://embodied-astra.github.io. Code, prompts and all episode logs: https://github.com/jackbjed/embodied-astra-robomme.
Sources
- Do As I Can, Not As I Say: Grounding Language in Robotic Affordances
- $\pi_0$: A Vision-Language-Action Flow Model for General Robot Control
- RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control
- Show-Harness: Just a VLM Agent Can Play Robots
- In-Context Robot Learning with VLM Agents
- PonderPounce: A Pretrained MLLM as an Episode Context Engine for Robot Control
- RoboMME: Benchmarking and Understanding Memory for Robotic Generalist Policies
- Transferring the Intelligence of VLMs to Robotic Control
- Inner Monologue: Embodied Reasoning through Planning with Language Models
- VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Models
- OCC4M: Object-Centric 4D Memory for Spatiotemporal Reasoning in Long-Horizon Manipulation
- OpenVLA: An Open-Source Vision-Language-Action Model
- RoboICL: Embodied In-Context Learning with GPT-6 Astra
- $\pi_{0.5}$: a Vision-Language-Action Model with Open-World Generalization
- An Unexpected Robot Policy: Early Evaluations of GPT-6 Astra on RoboDojo and Beyond
Related papers
- FMT x: An Efficient and Asymptotically Optimal Extension of the Fast Marching Tree for Dynamic Replanning
- MPCFormer: A physics-informed data-driven approach for explainable socially-aware autonomous driving
- RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies
- ARC: A Reasoning Recipe for Robot Foundation Models
- HRDexDB: A 4D Dexterous Grasping Dataset Across Human and Multiple Robot Embodiments
- APT: Action Expert Pretraining Improves Instruction Generalization of Vision-Language-Action Policies