Plans Are Not Executions: Adapting Frozen World-Model Planners to Controller-Induced Execution Shift

arXiv:2610.22041 · cs.RO · Submitted 2026-10-15 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.

Rosa: Today's paper: "Plans Are Not Executions".

Dev: The gist: ExecutionBridge, which infers controller-specific execution characteristics from planned–executed trajectory pairs and uses them to adapt the native ego latent rollout while keeping the original planned trajectory unchanged,

Rosa: First, who's behind it and why it matters.

Title and authors: Rosa: The paper is called "Plans Are Not Executions: Adapting Frozen World-Model Planners to Controller-Induced Execution Shift" by Danqi Zhao, Keyu Chen, Wenchao Sun, Minglei Huang, Jianhe Zhang, Yan Zhu, Hao Cheng, Zifan Cao and Sifa Zheng.

Dev: It’s interesting because the title itself is a direct statement: plans are not executions. The authors are looking at how different controllers can cause an execution shift that messes up the world model's predictions during deployment.

Taro: It sounds like they’re focusing on that moment right after you leave the simulation and hit the real road where things aren't exactly as you expected.

Rosa: Right. And what they show is that we can adapt the frozen world model planner by keeping the original planned trajectory as the input but using some learned controller-specific execution evidence to modify how it imagines that future, which they call adapting the latent rollout.

Dev: The big idea here is moving away from simply substituting executed trajectories for planned ones and instead adapting what's imagined based on how well a specific controller has behaved before.

Taro: So it’s about learning the signature of a controller’s execution style and using that signature to fine-tune the world model’s prediction mechanism without ever touching the high-level plan.

Rosa: Right. It suggests that instead of trying to make the execution match what was planned, we should use execution data to correct how well the model predicts what happens next for *that specific controller*.

The paper's summary: Dev: To summarize the paper, they introduce ExecutionBridge, which has two main parts: a Controller Encoder and a Latent Bridge. The Controller Encoder learns a compact representation of the controller that captures those persistent execution characteristics from the planned and executed trajectory pairs.

Rosa: That encoder doesn't need explicit labels or dynamics parameters; it just learns this representation by optimizing an auxiliary task where some execution decoder predicts the execution of held-out query trajectories for training supervision.

Taro: So they train this encoder to recognize *how* a specific controller tends to execute things, not just what the trajectory looks like in isolation.

Dev: And then, at deployment time, for any candidate trajectory, the frozen world model produces its native latent rollout conditioned on this learned controller representation.

Rosa: Then the Latent Bridge steps in to predict a bounded correction to that native ego latent using a formula modulated by a learned gate and projected onto some trust region of radius epsilon. This correction is injected into the ego-related component of the original rollout.

Taro: So it’s like they are taking what the world model *thinks* should happen for that plan and nudging it based on what we know about this specific controller's execution tendencies.

Dev: The main takeaway here is that by keeping the original planned trajectory as the conditioning input, you get better decision quality across unseen controller configurations because you are adapting the imagined future while preserving the semantic role of the plan for downstream control.

The paper's improvements: Rosa: What they show as an improvement is that ExecutionBridge consistently improves planning performance, measured by PDMS, across ten unseen controller configurations and multiple frozen world model planners.

Dev: The results are quite strong when you look at the low-score subset; the PDMS goes up from thirty-two point zero four to forty-six point one three, which is a gain of fourteen point zero nine points on that subset alone, which is substantial compared to what they found from just substituting executed trajectories.

Taro: It seems like the adaptation mechanism works well because it’s not just tweaking the geometric error; it actually improves the ranking of candidates in a way that matters for decision making.

Rosa: And they noted something interesting about how the method scales: it remains largely insensitive to the support set size, with planning performance staying nearly unchanged when you go from sixty-four pairs up to one thousand twenty-four pairs.

Dev: That’s important because it means we don't need a massive amount of paired data from deployment controllers just to get this adaptation mechanism working reliably. It suggests that the controller representation learned by the encoder is capturing enough persistent information.

Taro: So, if we can infer those characteristics from a relatively small set of examples, it’s much more practical for real-world deployment than needing huge datasets for training.

Conclusion: Rosa: To wrap up, the paper "Plans Are Not Executions: Adapting Frozen World-Model Planners to Controller-Induced Execution Shift" suggests that we should use execution evidence not to replace the plan, but to adapt how we imagine the future for a specific controller.

Dev: The core idea is that by inferring controller-specific execution characteristics via the Controller Encoder and using them in the Latent Bridge, you can achieve better decision quality across unseen controllers while keeping the original planned trajectory as our anchor.

Taro: I just want to add that this framework isn't tied to any specific world model architecture; it just adapts the latent representation conditioned on those learned controller characteristics, which makes it quite versatile for different setups.

Rosa: It really seems like a simple principle: preserve the plan’s semantic role and use execution evidence to adapt the future imagined for it. ExecutionBridge consistently improves planning across unseen controllers and multiple frozen world model planners.

Dev: It moves the focus from geometric accuracy of execution to semantic alignment between what we planned and how a specific controller actually behaves in motion.

Taro: It’s a useful tool for when you have a fixed plan but the vehicle dynamics or control system introduces unexpected execution shifts that break the model's assumptions.

Rosa: That’s it for this paper. We'll be looking at other work next time, but this one shows how to use execution data smartly to improve planning without changing the plan itself.

Danqi Zhao, Keyu Chen, Wenchao Sun, Minglei Huang, Jianhe Zhang, Yan Zhu, Hao Cheng, Zifan Cao

School of Vehicle and Mobility, Tsinghua University

cs.RO

Submitted: 2026-10-15

Updated: 2026-10-15

Comments: 8 pages, 7 figures, 4 tables

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 84/100

The gist: The gist: ExecutionBridge, which infers controller-specific execution characteristics from planned–executed trajectory pairs and uses them to adapt the native ego latent rollout while keeping the

Key concepts

Planning–Execution Gap: This occurs when a frozen world model predicts a future trajectory that doesn't match how a deployed controller actually drives. The model relies on training-time mappings, but the real-world execution is different, leading to an inconsistency between the imagined and actual motion.
ExecutionBridge: This framework infers unique execution traits for each controller from pairs of planned and executed trajectories. It uses these traits to adapt the world model's internal latent representation. This allows the planner to use a better-conditioned future prediction without changing the original plan itself.
Controller Encoder (CE): The CE learns a compact representation of each controller's execution style from support set samples. It is trained using an auxiliary task that predicts execution for held-out queries, helping it capture persistent, controller-specific behaviors needed for adaptation.

Summary

The gist: ExecutionBridge, which infers controller-specific execution characteristics from planned–executed trajectory pairs and uses them to adapt the native ego latent rollout while keeping the original planned trajectory unchanged, consistently improves planning across unseen controller configurations and multiple frozen worldmodel planners.

The Planning–Execution Gap

World-model planners evaluate candidate trajectories by predicting their future consequences, implicitly relying on the plan-to-motion relationship learned during training At deployment, a different downstream controller may execute the same planned trajectory differently, creating a planning–execution gap between the frozen model’s imagined future and actual execution This discrepancy arises because a frozen world model may predict a future that is consistent with its implicit training-time plan-tomotion mapping, yet inconsistent with the execution actually encountered by the deployed vehicle A natural remedy is to replace the planned trajectory with its predicted or observed execution However, we find that better execution fidelity does not necessarily yield better decisions; directly substituting predicted execution trajectories yields only a 0.048 PDMS improvement across 10 unseen controller configurations, far smaller than the gain from our proposed adaptation We attribute this failure to a semantic mismatch between planned and executed trajectories, where the executed trajectory already describes the motion resulting after downstream control

ExecutionBridge Framework

We introduce ExecutionBridge, which infers controller-specific execution characteristics from planned–executed trajectory pairs and uses them to adapt the native ego latent while keeping the original planned trajectory unchanged The framework consists of two lightweight components: a Controller Encoder (CE) and a Latent Bridge (LB) The Controller Encoder maps the support set Sc to a compact controller representation ec that captures persistent controller-specific execution characteristics This representation ec is learned using an auxiliary support–query execution-prediction task, where an auxiliary execution decoder predicts the execution of a held-out query trajectory for training supervision and is discarded at deployment

Controller Representation Learning

The Controller Encoder learns the controller representation without controller labels or explicit dynamics parameters by optimizing a loss function Lexec = ltraj τˆc j, τ c j, where ltraj measures trajectory position and heading errors Predicting the execution of unseen query trajectories encourages ec to capture controller-specific execution characteristics that persist across different planned trajectories, rather than memorizing individual support samples The Controller Encoder is then frozen after this training phase

Latent Adaptation Mechanism

For each candidate, the frozen world model first produces its native rollout from the original planned trajectory and extracts its ego latent Conditioned on ec, the Latent Bridge predicts a bounded correction to the native ego latent using an update formula that is modulated by a learned gate gϕ(·) and projected onto a trust-region of radius ϵ∥ξ∥2 The resulting controller-conditioned rollout is obtained by injecting this bounded correction into the ego-related latent component of the native rollout The planner, world model, and reward head remain frozen throughout the adaptation process

Evaluation and Results

We evaluate ExecutionBridge under a strict controller-disjoint protocol across ten unseen controller configurations ExecutionBridge consistently improves planning performance (PDMS) across these unseen controllers (Table II) The improvement is substantially larger on the low-score subset, where PDMS increases from 32.04 to 46.13 (+14.09) Figure 7 illustrates that better execution fidelity substantially reduces geometric error but only marginally improves candidate ranking ExecutionBridge yields the best decision quality by adapting the imagined future while preserving the plan as the world-model condition The method is largely insensitive to support-set size, with planning performance remaining nearly unchanged across N = 64 to 1024 This suggests that a small support set is sufficient to infer controller-specific execution characteristics

Conclusion

ExecutionBridge addresses the planning–execution gap by inferring controller-specific execution characteristics from planned–executed trajectory pairs and adapting the native ego latent rollout while retaining the original plan as the conditioning input These results support a simple principle: preserve the plan’s semantic role and use execution evidence to adapt the future imagined for it ExecutionBridge consistently improves planning across unseen controllers and multiple frozen worldmodel planners The proposed controller-conditioned adaptation is not tied to a WoTE-specific architecture Representative examples in Figure 4 further illustrate how the adapted evaluator leads to more reliable trajectory selection in challenging driving scenarios The method is largely insensitive to support-set size, with execution ADE and planning performance remaining nearly unchanged fro N = 64 to 1024 The proposed controller-conditioned adaptation is not tied to a WoTE-specific architecture Representative examples in Figure 4 further illustrate how the adapted evaluator leads to more reliable trajectory selection in challenging driving scenarios The method is largely insensitive to support-set size, with execution ADE and planning performance remaining nearly unchanged fro N = 64 to 1024 The proposed controller-conditioned adaptation is not tied to a WoTE-specific architecture Representative examples in Figure 4 further illustrate how the adapted evaluator leads to more reliable trajectory selection in challenging driving scenarios The method is largely insensitive to support-set size, with execution ADE and planning performance remaining nearly unchanged fro N = 64 to 1024 The proposed controller-conditioned adaptation is not tied to a WoTE-specific architecture Representative examples in Figure 4 further illustrate how the adapted evaluator leads to more reliable trajectory selection in challenging driving scenarios The method is largely insensitive to support-set size, with execution ADE and planning performance remaining nearly unchanged fro N = 64 to 1024 The proposed controller-conditioned adaptation is not tied to a WoTE-specific architecture Representative examples in Figure 4 further illustrate how the adapted evaluator leads to more reliable trajectory selection in challenging driving scenarios The method is largely insensitive to support-set size, with execution ADE and planning performance remaining nearly unchanged fro N = 64 to 1024 The proposed controller-conditioned adaptation is not tied to a WoTE-specific architecture Representative examples in Figure 4 further illustrate how the adapted evaluator leads to more reliable trajectory selection in challenging driving scenarios The method is largely insensitive to support-set size, with execution ADE and planning performance remaining nearly unchanged fro N = 64 to 1024 The proposed controller-conditioned adaptation is not tied to a WoTE-specific architecture Representative examples in Figure 4 further illustrate how the adapted evaluator leads to more reliable trajectory selection in challenging driving scenarios The method is largely insensitive to support-set size, with execution ADE and planning performance remaining nearly unchanged fro N = 64 to 1024 The proposed controller-conditioned adaptation is not tied to a WoTE-specific architecture Representative examples in Figure 4 further illustrate how the adapted evaluator leads to more reliable trajectory selection in challenging driving scenarios The method is largely insensitive to support-set size, with execution ADE and planning performance remaining nearly unchanged fro N = 64 to 1024 The proposed controller-conditioned adaptation is not tied to a WoTE-specific architecture Representative examples in Figure 4 further illustrate how the adapted evaluator leads to more reliable trajectory selection in challenging driving scenarios The method is largely insensitive to support-set size, with execution ADE and planning performance remaining nearly unchanged fro N = 64 to 1024 The proposed controller-conditioned adaptation is not tied to a WoTE-specific architecture Representative examples in Figure 4 further illustrate how the adapted evaluator leads to more reliable trajectory selection in challenging driving scenarios The method is largely insensitive to support-set size, with execution ADE and planning performance remaining nearly unchanged fro N = 64 to 1024 The proposed controller-conditioned adaptation is not tied to a WoTE-specific architecture Representative examples in Figure 4 further illustrate how the adapted evaluator leads to more reliable trajectory selection in challenging driving scenarios The method is largely insensitive to support-set size, with execution ADE and planning performance remaining nearly unchanged fro N = 64 to 1024 The proposed controller-conditioned adaptation is not tied to a WoTE-specific architecture Representative examples in Figure 4 further illustrate how the adapted evaluator leads to more reliable trajectory selection in challenging driving scenarios The method is largely insensitive to support-set size, with execution ADE and planning performance remaining nearly unchanged fro N = 64 to 1024 The proposed controller-conditioned adaptation is not tied to a WoTE-specific architecture Representative examples in Figure 4 further illustrate how the adapted evaluator leads to more reliable trajectory selection in challenging driving scenarios The method is largely insensitive to support-set size, with execution ADE and planning performance remaining nearly unchanged fro N

Improvements for AI systems

The system can improve planning performance across unseen controller configurations by adapting the native ego latent using controller-specific execution information via ExecutionBridge, which infers a compact controller representation ec that captures persistent controller-specific execution characteristics.
The improved AI system will mitigate the planning–execution gap by using the novel adaptation principle: execution information should adapt how the future is imagined, rather than replace the planned trajectory itself, by adapting the ego latent using controller-specific execution evidence.
The system will achieve superior decision quality in challenging driving scenes by utilizing a mechanism that supports controller-conditioned candidate evaluation while keeping the core planning components frozen, as demonstrated by consistently improving PDMS and TTC on unseen controllers.