Anchoring and Rescaling Attention for Semantically Coherent Inbetweening

arXiv:2603.17651 · cs.CV, cs.AI · Submitted 2026-03-18 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "Anchoring and Rescaling Attention for Semantically Coherent Inbetweening".

Jane: Here's the gist of the paper. The gist The authors propose training-free mechanisms, Keyframe-anchored Attention Bias and Rescaled Temporal RoPE, to improve generative inbetweening by providing semantic and temporal guidance from keyframes and text onto intermediate frames.

Tom: How it works The overall method pipeline is presented in Figure 3, which demonstrates the application of the approach on the First-Last-Frame-to-Video (FLF2V) pipeline in Wan2.1

3: .

Jane: To ensure pace stability and semantic fidelity, target anchors are designed to guide intermediate frames on the keyframes and text in the cross-attention (Sec.

Tom: 3.2) Furthermore, Rescaled Temporal RoPE is proposed as a simple adjustment to the self-attention layer of DiT block (Sec.

Title and authors: Tom: So we're looking at this new paper called "Anchoring and Rescaling Attention for Semantically Coherent Inbetweening". It's about fixing those issues where generative inbetweening gets messy when the keyframes are far apart or the motion is wild.

Jane: Right, so basically, they’ve designed a way to give intermediate frames better guidance by using information from those two end frames and the text prompt. It sounds like they’re trying to make sure what you see in between actually makes sense temporally and semantically.

Lu: What's interesting here is that the paper builds on models like Diffusion Transformer based video models, which can handle those two keyframes and text together, but it points out that as sequences get longer or the motion gets more dynamic, that guidance from the keyframes starts to weaken along the generation process.

Meng: I always wonder how much effort this takes in practice. If you're deploying this on a production system for generating video, adding extra attention mechanisms sounds like it adds complexity and inference time.

Lalam: It seems like they are proposing two main training-free tools to do this: Keyframe-anchored Attention Bias and Rescaled Temporal RoPE. That’s the core idea we need to focus on first.

Tom: Exactly, those are the two mechanisms that seem central to the whole approach described in "Anchoring and Rescaling Attention for Semantically Coherent Inbetweening". Let's talk about what they actually propose doing with these tools.

Jane: Okay, so let's unpack Keyframe-anchored Attention Bias first. The paper suggests this is implemented on cross-attention to maintain semantic fidelity and stable pacing. How does that translate for a listener?

Lu: They construct target attention anchors M(t) based on two keyframe anchors derived from averaging cross-attention maps from each condition—the two input keyframes and the text prompt—and they linearly interpolate these to get frame-wise target anchors.

Tom: That interpolation is what creates a small frame-wise logit bias, B(t), which gently steers the intermediate frames toward those keyframe anchors, and then they replace the original attention weights with a softmax of that guided attention.

Jane: So it’s like giving the AI a gentle tug toward where it *should* be based on what's already important in the first and last frames, plus what the text says. It’s supposed to improve semantic fidelity and pace stability because of that explicit cue.

Meng: I see how that addresses pacing issues, but how do we know if this bias is strong enough without overfitting to just those two keyframes?

Lalam: The paper also computes three symmetric cross-attentions—First ↔ video, Last ↔ video, and text ↔ video—each refined by its own M(t) and beta(t), which are then equally weighted and averaged.

Tom: So it’s not just one anchor guiding the frame; it’s a combination of anchors from different cross-attention relationships that gets fused together to guide the intermediate frames. It’s like having multiple directional arrows pointing toward the correct place.

Title and authors: Jane: And that sounds like it should help with consistency across different parts of the video, not just one static point in time, which is a big deal for making things look realistic.

Lu: Then there's the second part: Rescaled Temporal RoPE. This is introduced as a simple adjustment to the self-attention layer of the DiT block. They scale temporal RoPE near the keyframes up and reduce it in other frames.

Tom: That sounds like they are making edge frames place most of their attention on nearby frames while middle frames spread their attention across a wider temporal range. It’s about sharpening attention to preserve keyframe fidelity while broadening it so intermediate frames attend more stably to both keyframes.

Jane: So, instead of the RoPE doing the same thing everywhere, they are making the scaling dynamic based on where you are in the sequence—near the edges versus in the middle. It’s a clever way to manage temporal coherence across different parts of time.

Meng: If this works well for short sequences, I need to know how robust it is when we push it to longer sequences or more complex motions where those keyframe anchors might become less relevant over a long span.

Lalam: The paper addresses that by proposing Rescaled Temporal RoPE specifically to solve the challenge of vanilla RoPE lacking an explicit mechanism for anchoring the two keyframes, which leads to frame inconsistency.

Tom: It’s essentially tackling the problem where standard temporal rotation position embeddings don't inherently know which frames are the critical anchors we need to respect.

Jane: So, if we combine the attention bias guiding us toward semantics and Rescaled Temporal RoPE managing how time is represented in the model, what does that mean for generative inbetweening overall?

Lu: It means they’ve created a training-free approach that acts as a general plug-in to any video DiT models. The goal is to improve frame consistency, semantic fidelity, and pace stability across both short and long sequences.

Meng: That's ambitious. If it works without needing massive amounts of new training data for every specific video, that drastically lowers the cost of fine-tuning for a new domain.

Lalam: And they back up their claims with evaluations using TGI-Bench, which is a benchmark curated specifically for text-conditioned GI. They tested it across sequence lengths from twenty-five frames up to eighty-one frames.

Tom: Which is important because it shows the method isn't just theoretical; they tested it on challenging scenarios like dynamic motion and occlusion.

Jane: And looking at the results, they claim state-of-the-art frame consistency, semantic fidelity, and pace stability across those different challenges. They also show qualitative improvements in preserving semantically important objects consistently across frames thirty to forty <ref:2603.17651#pg1>.

Lu: The analysis on TGI-Bench shows that occlusion is the most difficult challenge for all models tested, and their method achieves the best LPIPS and VBench scores specifically on that challenge.

Meng: That’s a strong result for occlusion, which is notoriously hard for these kinds of models to handle correctly in real video generation. But what about speed? How much slower is this guided generation compared to just running the baseline model?

Title and authors: Tom: The paper focuses more on the quality metrics—consistency and fidelity—but the goal seems to be maintaining pace stability, which implies that if it’s faster than rerunning a failed generation, it’s a win.

Jane: So, to summarize this "Anchoring and Rescaling Attention for Semantically Coherent Inbetweening," they introduced Keyframe-anchored Attention Bias and Rescaled Temporal RoPE to guide intermediate frames using explicit keyframe information and better temporal scaling within the self-attention layer.

Lu: The implication is that these two mechanisms provide a robust, training-free way to inject semantic and temporal guidance directly into DiT architectures for generative inbetweening.

Meng: It’s a useful plug-in if you want to quickly improve the quality of existing models without retraining the whole thing from scratch, but we still have to worry about the computational overhead of calculating those anchors at inference time.

Lalam: And they point out a limitation: they focus on guiding frames toward the conditions, and they also note that occlusion is where all models struggle most, so while it performs well there, we need to see if it scales perfectly when things get even more chaotic than what’s in TGI-Bench.

Tom: So the paper gives us a set of tools—KAB and ReTRo—and a benchmark—TGI-Bench—to systematically diagnose where generative inbetweening is breaking down.

Jane: It seems like a solid contribution because it doesn't rely on massive new datasets but rather refines the existing attention mechanisms to handle the inherent difficulty of synthesizing plausible transitions between sparse keyframes.

Lu: Overall, the work suggests that by explicitly anchoring attention to known important points in time and scaling temporal representation appropriately, we can achieve better semantic coherence during generation.

Meng: If this method proves effective on real traffic—say, generating video for a specific brand or product where consistency is everything—it could really streamline our content creation pipeline.

Lalam: That’s the potential impact: making the output more reliable when you need precise temporal continuity between known start and end points.

Tom: So that wraps up our look at "Anchoring and Rescaling Attention for Semantically Coherent Inbetweening." It’s a solid piece of engineering work showing how targeted attention modifications can improve model behavior without requiring massive retraining.

Jane: Exactly. We’ve seen how they use the keyframe anchors and the temporal scaling to stabilize the generation process, especially when dealing with hard cases like occlusion in their TGI-Bench tests.

Lu: The future work would likely involve exploring how these anchor mechanisms generalize to even sparser keyframe sets or more complex scene dynamics that go beyond what's covered in those benchmarks.

Meng: From a deployment standpoint, the next step would be testing if this method can be integrated into a live inference pipeline efficiently enough to keep latency down for users.

Lalam: It seems like they’ve given us a clear path forward for improving the reliability of text-conditioned generative inbetweening by focusing on these specific attention interventions.

The paper's summary: Tom: So, to recap, they’re using two training-free tools—Keyframe-anchored Attention Bias and Rescaled Temporal RoPE—to make generative inbetweening better by giving intermediate frames explicit guidance from the keyframes and text prompt.

Jane: Right, so it’s about giving the AI a clear roadmap when it’s trying to fill in the gaps between two important moments in a video. It stops that gap from looking totally random or inconsistent.

Tom: Exactly. The paper shows how Keyframe-anchored Attention Bias creates these anchors by averaging attention maps from the start and end frames and then using those to gently nudge the intermediate frames toward what they should be attending to.

Jane: That sounds like a very smart way to inject semantic meaning into the generation process, making sure the object or action in between matches what was established at the keyframes.

Tom: And on top of that, Rescaled Temporal RoPE is adjusting how time itself is represented in the self-attention layer. They scale temporal RoPE up near those keyframes and down elsewhere to sharpen attention on the edges while spreading it out in the middle.

Jane: So, it’s like they’re telling the model: "Pay extra close attention to these critical points, but for everything else, look at a wider range of time." It addresses that problem where standard temporal methods just aren't aware of which frames matter most.

Tom: That’s what Lu was interested in—the idea that these mechanisms act like a general plug-in you can use on any video Diffusion Transformer model without retraining the whole thing.

Jane: Which is huge because it means we don't have to train a brand new model just to fix the way it handles tricky transitions between frames.

Tom: But they also didn't ignore the practical side of things, which is why they built TGI-Bench, a benchmark specifically for text-conditioned inbetweening.

Jane: That benchmark is really interesting because it systematically tests models across different kinds of video challenges, like dynamic motion or occlusion, and it shows where models actually fail.

Tom: And the results on that benchmark are pretty solid—they show state-of-the-art scores for frame consistency and semantic fidelity even on long sequences up to eighty one frames.

Jane: It’s a strong signal that these two techniques, when working together, really do improve the reliability of video generation in those hard cases they tested.

Tom: That leads us to thinking about what this means practically—if occlusion is the hardest thing for models, and this method handles it well on TGI-Bench, then we might see a real improvement in generating realistic action sequences where things block each other out.

The paper's improvements: Tom: So, to wrap up what they found—the improvements they proposed—they’re really showing how these two mechanisms work together to stabilize the generation process without needing new training data for every single video.

Jane: Right, so we already talked about how Keyframe-anchored Attention Bias and Rescaled Temporal RoPE fix specific problems like inconsistent pacing and temporal artifacts by giving the model explicit cues on what to pay attention to where it’s supposed to be.

Tom: What’s new in terms of improvement is that this isn't just a patch; it's designed as a general plug-in for any video Diffusion Transformer model you already have, which is what Lu was excited about.

Jane: And they also introduced TGI-Bench, which is their way of systematically diagnosing the weaknesses across different video challenges so researchers know exactly where the current AI models are struggling most.

Tom: The key improvement there is using that benchmark to pinpoint things like occlusion as the hardest problem for all existing models, and showing that this method performs best on it.

Jane: So, for a listener who doesn't read papers, it means this AI system can generate much more consistent video in between keyframes than before. It’s about making the "in-between" part look like it actually belongs to the start and end of the video.

Tom: And Lu thinks that if this approach generalizes well, it could really simplify how we fine-tune models for different types of videos because you don't have to relearn everything from scratch.

Jane: That makes sense. If we can use these attention mechanisms as a universal guidance system, it means the next generation of generative AI video won't need so much data just to handle complex temporal tasks.

Tom: It’s about making the existing architecture smarter through better attention control rather than just feeding it more raw video data.

Jane: So, while they don't claim a total solution for everything, they provide a concrete set of tools and a diagnostic test that shows exactly how these targeted interventions can boost quality in real-world scenarios.

Tom: The implication is that we can start using these concepts as standard components in video generation pipelines to get better consistency right out of the gate.

Jane: And since occlusion was their hardest test case, it suggests this approach might be particularly useful for generating action sequences where things are hiding each other out.

Conclusion: Tom: So we’re wrapping up on "Anchoring and Rescaling Attention for Semantically Coherent Inbetweening." Basically, they showed how Keyframe-anchored Attention Bias and Rescaled Temporal RoPE can fix pacing and consistency issues in generative inbetweening using just existing models.

Jane: It sounds like they’ve given us a practical way to inject semantic meaning into the generation process by anchoring the attention mechanisms to known important moments like keyframes.

Tom: Exactly, and they back it up with TGI-Bench, which tests it on hard cases like occlusion and shows that their method actually performs better there compared to other models.

Jane: So for someone just listening, this means video generation is going to look much smoother in the middle sections of a clip when you’re using these techniques. It changes how reliable AI-generated video feels in real life.

Tom: Lu was really interested in the fact that they presented this as a training-free approach, which means we don't have to spend weeks retraining models just to get these stability improvements on top of what we already have.

Jane: That’s a big deal because it lowers the barrier for researchers who want to improve video AI without needing massive new datasets.

Tom: Meng, you mentioned deployment earlier—does this sound like something that would actually run well on a production system?

Meng: I'm skeptical about the scale. If calculating those anchors and biases adds significant overhead to inference, it might not be practical for high-throughput video generation pipelines right now.

Jane: That’s a fair concern, Meng, because the paper focuses heavily on the quality gains rather than the speed trade-off in milliseconds.

Tom: It does focus on quality—they show state-of-the-art scores on TGI-Bench for frame consistency and semantic fidelity across those longer sequences.

Lu: I think what’s interesting is how they structured the Rescaled Temporal RoPE to handle the temporal frequency rows differently at the edges versus the middle, which is a clever way to manage that temporal spread.

Jane: And Lalam, you look at this from a literature standpoint—how does this fit with what we know about attention sinks or logit interventions?

Lalam: I see this as a refinement of those ideas; it’s essentially an inference-time intervention that steers the model's self-attention strategy based on temporal location, which is pretty sure will make a difference in coherence.

Tom: So to summarize, "Anchoring and Rescaling Attention for Semantically Coherent Inbetweening" gives us two ways—Attention Bias and RoPE scaling—to stabilize video generation using keyframe data without needing new training.

Jane: It’s a solid piece of work that shows how targeted attention modifications can significantly improve the reliability of text-conditioned generative inbetweening.

Tom: It definitely provides a clear path forward for making those transitions between start and end frames look way more plausible.

Jane: Next time, we’ll look at how models are handling domain shift in video, because that’s another huge challenge for current AI systems.

Tae Eun Choi, Sumin Shim, Junhyeok Kim, Seong Jae Hwang

Yonsei University

cs.CV, cs.AI

Submitted: 2026-03-18

Updated: 2026-10-20

Comments: Accepted to CVPR 2026; Code is released at https://github.com/teunchoi/TGI

Journal ref: Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8976-8985, 2026

Code: https://github.com/teunchoi/TGI

License: http://creativecommons.org/licenses/by-nc-nd/4.0/

Importance score: 92/100

The gist: The gist The authors propose training-free mechanisms, Keyframe-anchored Attention Bias and Rescaled Temporal RoPE, to improve generative inbetweening by providing semantic and temporal guidance from

Key concepts

Keyframe-anchored Attention Bias (KAB): This mechanism uses information from the first and last frames' cross-attention to create 'keyframe anchors.' These anchors are then used to add a small logit bias to intermediate frames, gently steering them toward the semantic guidance provided by the keyframes and text prompt. This aims to maintain semantic fidelity.
Rescaled Temporal RoPE (ReTRo): ReTRo modifies the temporal position embedding (RoPE) in self-attention layers. It increases the RoPE scale near edge frames to sharpen attention on those frames, ensuring keyframe fidelity. Conversely, it reduces the scale in the middle of the sequence to allow intermediate frames to attend more stably across a wider temporal range.
TGI-Bench: This is a curated benchmark designed to systematically test models for text-conditioned generative inbetweening. It pairs videos with textual prompts and classifies them into four challenges: dynamic motion, linear motion, occlusion, or near-static. This allows researchers to diagnose specific model weaknesses.
Generative Inbetweening: This refers to the process of generating coherent video frames that smoothly transition between two existing keyframes or a start and end frame. The goal is to create realistic intermediate frames that maintain both semantic consistency (what is happening) and temporal stability (how things move over time).

Terminology

Consistency — 2× in this paper · explained in 3 episodes
Benchmark — 1× in this paper · explained in 2 episodes
Fidelity — 1× in this paper · explained in 2 episodes

Summary

The gist The authors propose training-free mechanisms, Keyframe-anchored Attention Bias and Rescaled Temporal RoPE, to improve generative inbetweening by providing semantic and temporal guidance from keyframes and text onto intermediate frames.

How it works

The overall method pipeline is presented in Figure 3, which demonstrates the application of the approach on the First-Last-Frame-to-Video (FLF2V) pipeline in Wan2.1 [3]. To ensure pace stability and semantic fidelity, target anchors are designed to guide intermediate frames on the keyframes and text in the cross-attention (Sec. 3.2) Furthermore, Rescaled Temporal RoPE is proposed as a simple adjustment to the self-attention layer of DiT block (Sec. 3.3). This mechanism increases the temporal RoPE scale near the keyframes and reduces it in other frames, which sharpens attention to maintain keyframe fidelity while broadening attention so intermediate frames attend more stably to both keyframes.

Keyframe-anchored Attention Bias (KAB)

Keyframe-anchored Attention Bias (KAB) is implemented on cross-attention to maintain semantic fidelity and stable pacing by gaining keyframe anchors from the keyframes’ cross-attention which are used to attract the intermediate frames towards the conditions: the two input keyframes and text prompt. The process involves constructing target attention anchors M(t) based on two keyframe anchors, where M(t) is linearly interpolated between A¯(0) and A¯(f−1), which are derived by averaging cross-attention maps from each condition to obtain target anchors A¯(0) and A¯(f−1). These anchors are then used to add a small frame-wise logit bias B(t) to the original cross-attention logits, defined as Le(t)h:=L(t)h + β(t)B(t), which gently steers the intermediate frames towards them. Finally, the original attention weights Ah are replaced with A˜h = softmaxL˜h, a conservative pull that preserves the model’s local patterns while guiding its global token allocation toward the keyframes and text prompt. The method also computes three symmetric cross-attentions: Ifirst ↔ video, Ilast ↔ video, and text ↔ video, each refined by its own M(t) and β(t), which are then equally weighted and averaged.

Rescaled Temporal RoPE (ReTRo)

Rescaled Temporal RoPE (ReTRo) is introduced to address the challenge that vanilla RoPE lacks an explicit mechanism that anchors the two keyframes, which leads to frame inconsistency. ReTRo is inspired by RoPE scaling methods for LLMs [5, 29] and involves constructing per-axis frequency rows Ψ(t, h, w). The temporal frequency row per frame is scaled to maintain edge frames fidelity and stabilize the middle of the sequence by defining s(t):= (sedge, t ∈ Tedge, smid, t ∈ Tmid). This results in Q˜ = RoPE(Q; ΨReTRo), K˜ = RoPE(K; ΨReTRo), where edge frames place most of their attention on nearby frames while middle frames spread their attention across a wider temporal range.

TGI-Bench and Evaluation

To evaluate models supporting text guidance, the authors introduce TGI-Bench, which curates sequences tailored for text-conditioned GI and pairs each with an aligned textual prompt. TGI-Bench consists of two keyframes, the corresponding ground-truth intermediate frames, a textual description, and its designated challenge category. It covers both short and long sequences with four sequence-length variants for 25, 33, 65, and 81 frames. Data curation involves selecting videos from the DAVIS [31] dataset as well as from the Pexels and Pixabay websites. Videos are then subsampled and provided to GPT-4.1 [12], which is prompted to generate a text description of the video and classify it into one of four challenge types: dynamic motion, occlusion, linear motion, or near-static.

Results and Analysis

Quantitative evaluation on 65- and 81-frame results shows that the authors' method achieves state-of-the-art frame consistency, semantic fidelity, and pace stability for both short and long sequences across diverse challenges using TGI-Bench. In qualitative comparisons, the method outperforms prior works in all three target challenges: semantic fidelity, pace stability and frame consistency. For instance, KAB preserves a semantically important object consistently across frames 30 to 40, while ReTRo removes the artifact visible around frame 20. The TGI-Bench analysis reveals that occlusion emerges as the most difficult for all models, and our method achieves the best LPIPS and VBench scores on this challenge.

Conclusion

In this paper, the authors propose two training-free mechanisms for generative inbetweening that can act as a general plug-in to any video DiT models. First, Keyframe-anchored Attention Bias (KAB) utilizes the model’s own cross-attention layer on the first and last frames as signals and guide intermediate frames toward a per-frame interpolation of these keyframe anchors. Furthermore, they incorporate Rescaled Temporal RoPE (ReTRo) which enlarges temporal RoPE scale at the edge frames and reduces in the middle to enhance frame consistency. Finally, they release TGI-Bench, a benchmark that systematically diagnoses the current challenges of generative inbetweening. They expect this benchmark to accelerate the field of text-conditioned GI, especially under dynamic motion and occlusion. The work is supported in part by the IITP RS-2024-00457882 (AI Research Hub Project), IITP 2020-II201361, NRF RS-2024-00345806, NRF RS23-17651v1, and RQT-25-1765.

--- Page 9 ---

Acknowledgement. This work was supported in part by the IITP RS-2024-00457882 (AI Research Hub Project), IITP 2020-II201361, NRF RS-2024-00345806, NRF RSRS 23-17651v1, and RQT-25-1765. Affiliations: Department of Artificial Intelligence (T.E.C, J.K, S.J.H), Department of Computer Science (S.S).

--- Page 10 ---

time Pos Self-attention Map time wedge f - wedge Rescaled Temporal RoPE text prompt Self-attention Video Keys edge frame frame 0 frame 1 interpolated Anchors Original Attention Logit Softmax & Guided Attention Logit L(t)h Le(t)h (t) =L(t)h + β(t)B(t)=L(t)h + β(t)log(M(f−2)+ε − logA¯[f−2]+ε, heads lq lk

--- Page 11 ---

Figure 3 Overall Pipeline of Our Method. Our model is built upon a video DiT pipeline that consists of DiT blocks with self-attention and cross-attention layers. Left: Keyframe-anchored Attention Bias is performed for each condition’s cross-attention, which aggregates crossattention maps from each keyframes to form keyframe anchors. These keyframe anchors are interpolated to frame-wise target anchors, which are used as a small logit bias to guide each intermediate frames. Right: Furthermore, we introduce Rescaled Temporal RoPE, which increases temporal RoPE scale at the edges and reduces in the middle.

--- Page 12 ---

Cross-attention Editing. Cross-attention has continuously been used as a control handle for text-driven image editing. Prompt-to-Prompt [16] preserves structure by copying and blending cross-attention maps across prompts and timesteps, while Attend-and-Excite [3] reweights under-attended tokens to mitigate missing-object failures.

--- Page 13 ---

Triple Isolated Cross-attention. In the baseline FLF2V pipeline, the model applies cross-attention to Ifirst alone, while Ilast and text are concatenated and attended jointly, yielding an asymmetric fusion across the three conditions.

--- Page 14 ---

(a) Example frames First frame Last frame (b) Statistics 25 frames 33 frames (c) 65 frame (d) 81 frame Figure S4 TGI-Bench. (a) One example from each challenge of our TGI-Bench is presented.

--- Page 15 ---

(b) The distribution of challenges according to the number of frames is illustrated Figure S4.

--- Page 16 ---

(a)

Improvements for AI systems

Keyframe-anchored Attention Bias (KAB) will guide intermediate frames using keyframe anchors from the keyframes’ cross-attention which are used to attract the intermediate frames towards the conditions: the two input keyframes and text prompt. This mechanism is expected to improve semantic fidelity and pace stability by providing explicit semantic and temporal cues.
Rescaled Temporal RoPE (ReTRo) will be implemented in self-attention layers, where it increases the temporal RoPE scale near the keyframes and reduces it in other frames, which allows intermediate frames to attend more stably to both keyframes while sharpening attention on edges to preserve keyframe fidelity.
The TGI-Bench benchmark will enable challenge-specific diagnosis of model strengths and weaknesses across four categories: dynamic motion, linear motion, occlusion, and near-static. This allows researchers to identify specific failure modes where models struggle, such as the most difficult challenge being occlusion.

Abstract

Generative inbetweening (GI) seeks to synthesize realistic intermediate frames between the first and last keyframes beyond mere interpolation. As sequences become sparser and motions larger, previous GI models struggle with inconsistent frames with unstable pacing and semantic misalignment. Since GI involves fixed endpoints and numerous plausible paths, this task requires additional guidance gained from the keyframes and text to specify the intended path. Thus, we give semantic and temporal guidance from the keyframes and text onto each intermediate frame through Keyframe-anchored Attention Bias. We also better enforce frame consistency with Rescaled Temporal RoPE, which allows self-attention to attend to keyframes more faithfully. TGI-Bench, the first benchmark specifically designed for text-conditioned GI evaluation, enables challenge-targeted evaluation to analyze GI models. Without additional training, our method achieves state-of-the-art frame consistency, semantic fidelity, and pace stability for both short and long sequences across diverse challenges.