Sometimes You Gotta Run Before You Can Walk: Run-then-Walk Scheduling Strategy for VLM Autonomous Driving
cs.RO, cs.CV
Submitted: 2026-09-22
Updated: 2026-10-07
License: http://creativecommons.org/licenses/by/4.0/
The gist: Recent VLM-based autonomous driving planners adopt GRPO-style reinforcement learning to optimize driving performance.
Terminology
Abstract
Recent VLM-based autonomous driving planners adopt GRPO-style reinforcement learning to optimize driving performance. However, existing GRPO recipes either optimize driving efficiency, risking progress-seeking but unsafe behavior, or enforce early safety constraints, leading to overly conservative behavior; both require lengthy training. To solve these problems, we first reveal two distinct RL regimes: a progress regime (Run-GRPO) that aggressively explores high progress, and a safety regime (Walk-GRPO) that restores safety under stable progress. Based on this finding, we propose Run-then-Walk, a simple yet effective two-stage reward scheduling strategy for GRPO, achieving both better performance and faster convergence. Unlike one-stage RL, which may focus on progress, safety, or a mixture of both within a single training phase, this schedule explicitly separates progress discovery from safety repair. In the Run phase, we focus on progress, allowing the policy to escape the conservative bias and discover high-progress modes. In the subsequent Walk phase, we introduce endpoint and safety strategy to repair unsafe behaviors from the Run phase. This reversed schedule overcomes the conservatism of Walk-first methods and the unsafe progress-seeking of joint optimization. We validate it with various VLM-based planners on multiple benchmarks: NAVSIMv1, NAVSIMv2, Navhard, and nuScenes. Extensive experiments demonstrate improved driving performance while requiring 40--50% fewer RL training epochs than the baselines. Code is available at https://github.com/haha-yuki-haha/AutoDrive-P3 with Run-then-walk.
Sources
- Pseudo-Simulation for Autonomous Driving
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- DriveFuture: Future-Aware Latent World Models for Autonomous Driving
- Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models
- EMMA: End-to-End Multimodal Model for Autonomous Driving
- AlphaDrive: Unleashing the Power of VLMs in Autonomous Driving via Reinforcement Learning and Reasoning
- Enhancing End-to-End Autonomous Driving with Latent World Model
- DriveVLA-W0: World Models Amplify Data Scaling Law in Autonomous Driving
- End-to-End Driving with Online Trajectory Evaluation via BEV World Model
- ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Driving
- Hydra-MDP: End-to-end Multimodal Planning with Multi-target Hydra-Distillation
- DriveWorld-VLA: Unified Latent-Space World Modeling with Vision-Language-Action for Autonomous Driving
- GPT-Driver: Learning to Drive with GPT
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- DeepSeekMath-V2: Towards Self-Verifiable Mathematical Reasoning
- InsightDrive: Insight Scene Representation for End-to-End Autonomous Driving
- DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Models
- Latent-WAM: Latent World Action Modeling for End-to-End Autonomous Driving
- VLM-AD: End-to-End Autonomous Driving through Vision-Language Model Supervision
- AutoDrive-R$^2$: Incentivizing Reasoning and Self-Reflection Capacity for VLA Model in Autonomous Driving
Related papers
- FMT x: An Efficient and Asymptotically Optimal Extension of the Fast Marching Tree for Dynamic Replanning
- MPCFormer: A physics-informed data-driven approach for explainable socially-aware autonomous driving
- RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies
- ARC: A Reasoning Recipe for Robot Foundation Models
- HRDexDB: A 4D Dexterous Grasping Dataset Across Human and Multiple Robot Embodiments
- APT: Action Expert Pretraining Improves Instruction Generalization of Vision-Language-Action Policies