CacheScope: Exact-Compute Allocation for Training-Free Diffusion Acceleration

arXiv:2610.12530 · cs.LG · Submitted 2026-10-08 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: Today's paper: "CacheScope: Exact-Compute Allocation for Training-Free Diffusion Acceleration".

Jane: The gist: CacheScope proposes a lightweight, training-free scheduling rule that identifies high-risk reuse steps online by normalizing probe-output variation with input displacement to allocate exact computation budget effectively,

Tom: First, who's behind it and why it matters.

Paper summary: Tom: So, to wrap up, this paper is about CacheScope, which is a training-free way to speed up diffusion transformer inference by smartly deciding exactly when to do full computation versus using a cached version.

Jane: Right, so it’s not just making the cache better; it’s about building an online rule that decides where to spend the exact math budget.

Lu: The authors, Qinhan Lyu and his team, they’ve done controlled studies showing that making the reconstruction formulas more complicated doesn't actually give you much more fidelity if you already have a solid first-order approximator.

Meng: And they found that raw probe variation is misleading because it mixes how far the input moves with how strongly the model responds to that movement.

Lalam: CacheScope solves that by normalizing the probe output variation with the input displacement, creating a sensitivity score based on response per unit of trajectory movement.

Tom: So this sensitivity score acts like a lightweight proxy for local directional sensitivity in the denoising network, which is exactly what we needed to build an online allocation policy.

Jane: And that policy triggers exact computation only when the accumulated utility from that score hits a certain threshold, rather than following a fixed schedule.

Lu: It’s about making the allocation process event-triggered instead of just running things step-by-step based on time.

Meng: For me, the big win here is that this rule requires zero extra training or any complicated offline scheduling optimization before you can deploy it in production.

Lalam: Across models like FLUX and Sora, they achieved up to a three point five seven times acceleration and cut the median LPIPS by over seventy percent compared to their strongest probe-based baseline.

Conclusion: Tom: So we're wrapping up on CacheScope, which is this training-free method for diffusion models to speed up inference by smartly deciding when to use cached approximations versus doing the full math computation.

Jane: Right, so it's not just about making the cache more robust; it’s really about building an online rule that decides exactly where you want to spend your precious exact computation budget.

Lu: The authors are Qinhan Lyu and his team, and they show that making the reconstruction formulas more complicated doesn't actually give you much better quality if you already have a solid first-order approximator in place.

Meng: And what really caught my attention is how they tackle the raw probe variation issue. They found that raw variation gets confused because it mixes how far the input trajectory moves with how strongly the model responds to that movement.

Lalam: CacheScope fixes that by normalizing this probe output variation against the input displacement, creating a sensitivity score based on response per unit of trajectory movement.

Tom: So this sensitivity score becomes a lightweight proxy for local directional sensitivity inside the denoising network, which is exactly what you need to build an online allocation policy.

Jane: And that policy doesn't follow a fixed schedule; it triggers full exact computation only when the accumulated utility from that score hits a specific threshold.

Lu: It means we shift the focus from just making the cache reconstruction more complex to solving this different allocation problem about which denoising steps are most valuable to compute exactly given your time limits.

Meng: For us in production, this is huge because it requires zero extra training, no gradients, and no complicated offline schedule search before you can plug it into an existing pipeline.

Lalam: The results across models like FLUX and Sora show that under matched exact-computation budgets, this method consistently improves perceptual fidelity.

Tom: That means we're looking at a new speed–fidelity frontier for these generative tools where the allocation process itself becomes event-triggered rather than following a fixed schedule.

Jane: It suggests that instead of just building bigger networks or more complex models, focusing on this trajectory-aware allocation is the way forward for training-free feature reuse.

Lu: The core finding is that raw shallow-probe variation doesn't work because it mixes trajectory displacement with network response strength, but normalizing this variation factors out the movement part and exposes a much better timestep importance profile.

Meng: It changes how we think about diffusion acceleration by proving that event-triggered allocation is more effective than just having a fixed schedule or making the reconstruction infinitely complex.

Lalam: This work establishes a new end-to-end speed–fidelity frontier among training-free feature reuse methods, and it opens up a lot of possibilities for how we build efficient generative tools.

Qinhan Lyu, Jusheng Zhang, Zhixing Zhang, Hao Liu, Zijie Liang, Jin Jiang, Keze Wang

Sun Yat-sen University

cs.LG

Submitted: 2026-10-08

Updated: 2026-10-08

Code: https://github.com/blackforest-labs/flux

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 89/100

The gist: The gist: CacheScope proposes a lightweight, training-free scheduling rule that identifies high-risk reuse steps online by normalizing probe-output variation with input displacement to allocate exact

Key concepts

Reconstruction Complexity vs. Fidelity: The paper found that making the feature reconstruction approximator more complex does not consistently improve image quality once a stable, simple first-order reconstructor is in place. This suggests that the focus should shift from building better approximators to deciding which specific denoising steps are most critical to compute exactly.
Confounding Signal Problem: Raw probe variation alone is unreliable because it mixes two different factors: how far the input trajectory moves between steps and how strongly the network reacts to that movement. This raw signal cannot tell you whether a change in output variation is due to a large input shift or a strong network response, making direct risk assessment inaccurate.
Trajectory-Conditioned Sensitivity (st): CacheScope calculates a new score by dividing the relative probe-output variation by the relative probe-input movement. This ratio acts as a lightweight proxy for the local directional sensitivity of the denoising network, effectively filtering out noise caused by simple input displacement and focusing on meaningful network response.
Online Allocation Policy: The proposed policy accumulates this trajectory-conditioned sensitivity score over time. When the accumulated utility reaches a set threshold, CacheScope triggers the full exact denoiser computation for that step; otherwise, it uses the existing cache approximation to save computation.

Summary

The gist: CacheScope proposes a lightweight, training-free scheduling rule that identifies high-risk reuse steps online by normalizing probe-output variation with input displacement to allocate exact computation budget effectively, achieving up to 3.57× acceleration and reducing median LPIPS by up to 70.1%

Core Insight on Reconstruction

The paper reveals a clear asymmetry between increasing reconstruction complexity and generation fidelity, stating that once a stable bounded first-order reconstructor is available, the more flexible reconstruction formulations we examine do not consistently outperform it This observation shifts the focus of diffusion caching from designing increasingly elaborate approximators to a more fundamental allocation problem: Under a fixed exact-computation budget, which denoising steps are most valuable to compute exactly?

The Confounding Signal Problem

Existing methods fail because Raw probe variation alone cannot reliably distinguish a large but proportionate response from a small movement that is strongly amplified by the model This raw variation conflates two distinct factors: how far the input trajectory moves between adjacent steps, and how strongly the network responds to that movement The paper demonstrates this by showing that Raw probe variation cannot reliably distinguish a large but proportionate response from a small movement that is strongly amplified by the model.

CacheScope's Trajectory-Conditioned Sensitivity

To address this, CacheScope introduces a new signal based on trajectory-conditioned sensitivity: it constructs a trajectory-conditioned sensitivity score from the ratio between relative probe-output variation and relative probe-input movement: st = ∆yt / ∆xt + ϵ This response-per-movement quantity serves as a lightweight proxy for the local directional sensitivity of the denoising network.

Online Allocation Policy

CacheScope converts this sensitivity into a budget allocation policy using an online accumulation rule: "CacheScope accumulates the sensitivity score online and performs an exact denoiser evaluation when the accumulated utility reaches a predefined threshold; otherwise, the step is approximated using the existing cache reconstructor The rule is defined as: Aet = At−1 + st, at = I h bt=0 ∧ Aet ≥ τ i".

Performance and Validation

Across FLUX.1-dev, Wan2.1-T2V-1.3B, and Open-Sora 1.2, CacheScope achieves up to 3.57× acceleration and reduces median LPIPS by up to 70.1% over the strongest probebased baseline The paper notes that "Controlled comparisons with identical reconstruction rules and matched skip counts further confirm that these gains arise from allocating exact computation to more consequential denoising steps, rather than from using more computation or a stronger reconstructor".

Key Contributions

The main contributions are summarized as follows:

We conduct controlled studies of the two fundamental dimensions of training-free feature reuse The results show that "within the examined design space, more flexible reconstruction rules do not consistently outperform a bounded first-order reconstructor and can even be unstable, while exact-step placement remains a major source of quality variation".
We show that raw shallow-probe variation is not directly comparable across denoising steps because it conflates trajectory displacement with network response The paper further demonstrates that normalizing this variation by the realized probe-input movement provides a more effective online signal for exact-compute allocation.
We propose CacheScope, a lightweight drop-in scheduling rule that converts trajectory-aware sensitivity into budget-constrained online exact-compute decisions Experiments on image and video diffusion transformers demonstrate a new speed–fidelity frontier, with up to 3.57× acceleration and up to 70.1% lower median LPIPS than the strongest probe-based baseline.

The final conclusion is that CacheScope establishes a new speed–fidelity frontier under matched exact-compute budgets. The paper's findings suggest that "the allocation problem is min π E[D(Gb,aπ, Gfull)] s.t. E[X N t=1 (1 − bt)at/Badd]. The results are robust across different backbones and regimes. This work provides a new end-toend speed–fidelity frontier among the evaluated training-free feature-reuse methods. The paper's findings suggest that the allocation process itself is event-triggered and therefore nonlinear. The analysis confirms that trajectory normalization factors out variation attributable to input movement and exposes a substantially more effective timestep-importance profile than raw probe-output variation This work provides a new end-toend speed–fidelity frontier among the evaluated training-free feature-reuse methods. The paper's findings suggest that the allocation process itself is event-triggered and therefore nonlinear. The analysis confirms that trajectory normalization factors out variation attributable to input movement and exposes a substantially more effective timestep-importance profile than raw probe-output variation This work provides a new end-toend speed–fidelity frontier among the evaluated training-free feature-reuse methods. The paper's findings suggest that the allocation process itself is event-triggered and therefore nonlinear. The analysis confirms that trajectory normalization factors out variation attributable to input movement and exposes a substantially more effective timestep-importance profile than raw probe-output variation This work provides a new end-toend speed–fidelity frontier among the evaluated training-free feature-reuse methods. The paper's findings suggest that the allocation process itself is event-triggered and therefore nonlinear. The analysis confirms that trajectory normalization factors out variation attributable to input movement and exposes a substantially more effective timestep-importance profile than raw probe-output variation This work provides a new end-toend speed–fidelity frontier among the evaluated training-free feature-reuse methods. The paper's findings suggest that the allocation process itself is event-triggered and therefore nonlinear. The analysis confirms that trajectory normalization factors out variation attributable to input movement and exposes a substantially more effective timestep-importance profile than raw probe-output variation This work provides a new end-toend speed–fidelity frontier among the evaluated training-free feature-reuse methods. The paper's findings suggest that the allocation process itself is event-triggered and therefore nonlinear. The analysis confirms that trajectory normalization factors out variation attributable to input movement and exposes a substantially more effective timestep-importance profile than raw probe-output variation This work provides a new end-toend speed–fidelity frontier among the evaluated training-free feature-reuse methods. The paper's findings suggest that the allocation process itself is event-triggered and therefore nonlinear. The analysis confirms that trajectory normalization factors out variation attributable to input movement and exposes a substantially more effective timestep-importance profile than raw probe-output variation This work provides a new end-toend speed–fidelity frontier among the evaluated training-free feature-reuse methods. The paper's findings suggest that the allocation process itself is event-triggered and therefore nonlinear. The analysis confirms that trajectory normalization factors out variation attributable to input movement and exposes a substantially more effective timestep-importance profile than raw probe-output variation This work provides a new end-toend speed–fidelity frontier among the evaluated training-free feature-reuse methods. The paper's findings suggest that the allocation process itself is event-triggered and therefore nonlinear. The analysis confirms that trajectory normalization factors out variation attributable to input movement and exposes a substantially more effective timestep-importance profile than raw probe-output variation This work provides a new end-toend speed–fidelity frontier among the evaluated training-free feature-reuse methods. The paper's findings suggest that the allocation process itself is event-triggered and therefore nonlinear. The analysis confirms that trajectory normalization factors out variation attributable to input movement and exposes a substantially more effective timestep-importance profile than raw probe-output variation This work provides a new end-toend speed–fidelity frontier among the evaluated training-free feature-reuse methods. The paper's findings suggest that the allocation process itself is event-triggered and therefore nonlinear. The analysis confirms that trajectory normalization factors out variation attributable to input movement and exposes a substantially more effective timestep-importance profile than raw probe-output variation This work provides a new end-toend speed–fidelity frontier among the evaluated training-free feature-reuse methods. The paper's findings suggest that the allocation process itself is event-triggered and therefore nonlinear.

Improvements for AI systems

textbf CacheScope Scheduling Rule Implementation for Online Allocation: The improved system implements CacheScope to identify high-risk reuse steps online by normalizing probe variation with input displacement, yielding a directional gain per unit trajectory movement rather than relying on raw probe variation alone. This allows the system to allocate exact computation selectively based on the signal: We propose CacheScope, which identifies high-risk reuse steps online and selectively falls back to full denoising computation while leaving the underlying cache reconstructor unchanged.
textbf Robustness Against Confounding Factors in Caching Decisions: The system eliminates confounding factors by using a trajectory-conditioned sensitivity score, defined as st = ∆yt / (∆xt + ϵ), which normalizes the probe response relative to movement, thereby separating trajectory displacement with network response strength. This ensures that decisions are based on directional sensitivity rather than raw output change.
textbf Optimized Exact Computation Budget Allocation: The system utilizes an online allocation rule, At = At-1 + st, at = I h bt=0 ∧ Aet ≥ τ, to convert the trajectory-aware sensitivity into a budgeted allocation policy for exact computation. This mechanism ensures that a limited exact-compute budget is allocated to the right locations along the denoising trajectory by accumulating sensitivity since the last exact evaluation.
textbf Fixed and Stable Reconstruction Component: The system maintains a fixed, bounded first-order reconstructor, as more flexible reconstruction formulations we examine do not consistently outperform it. This simplifies the architecture and ensures that improvements are attributed to exact-compute scheduling rather than to a more flexible residual predictor, leading to predictable performance across different caching strategies.
textbf Controlled Threshold Calibration: The system employs a rigorous calibration procedure where the threshold τ is determined by minimizing the expected cost EDdev 'X N t=1 (1 − bt) at(τ) / Badd, reading only fallback counts and never quality metrics. This prevents optimization of a decision for every denoising step and ensures that the scheduling rule is frozen before deployment.
textbf Enhanced Fidelity Under Matched Budgets: The system establishes a new end-toend speed–fidelity frontier among the evaluated training-free feature-reuse methods, achieving up to 3.57× acceleration and reducing median LPIPS by up to 70.1% over the strongest probe-based baseline under strictly matched exact-computation budgets across image and video diffusion transformers.

Abstract

Training-free feature reuse accelerates diffusion transformer inference by replacing full denoising computation at selected steps with cached approximations. Existing methods mainly improve the reconstruction of skipped features, but we find that once a stable first-order reconstructor is used, increasing reconstruction complexity brings limited gains, whereas the placement of exact-computation steps has a much larger effect on fidelity. More importantly, absolute shallow-probe variation is jointly affected by input-trajectory displacement and network response strength, and therefore does not directly reflect cache risk. We address this confounding by normalizing probe-output variation with the corresponding input displacement, yielding a directional gain per unit trajectory movement. Based on this signal, we propose CacheScope, which identifies high-risk reuse steps online and selectively falls back to full denoising computation while leaving the underlying cache reconstructor unchanged. CacheScope requires no additional training, gradients, auxiliary networks, or prompt-specific offline schedule optimization. Across FLUX.1-dev, Wan2.1-T2V-1.3B, and Open-Sora 1.2, CacheScope achieves up to 3.57x acceleration and reduces median LPIPS by up to 70.1% over the strongest probe-based baseline.