Not Just a Subject: Capturing Adaptive Scene-Specific Variance for Visual Storytelling

arXiv:2610.22320 · cs.CV · Submitted 2026-10-15 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "Not Just a Subject".

Jane: The gist: EOS-Story is a training-free framework that harmonizes identity consistency with scene-specific adaptation across text and vision spaces.

Tom: First, who's behind it and why it matters.

Title and authors: Tom: So this paper, "Not Just a Subject," basically shows how you can make image generation models handle scene changes without needing to retrain them for every new scenario.

Jane: Right, so what I get is that they’ve built this framework called EOS-Story that tries to keep the subject consistent even when the background or setting completely flips.

Tom: Exactly, they use two main ideas—Cross-Contextual Bridging and Adaptive Identity Preservation—to do this without any training.

Jane: Cross-Contextual Bridging is how they handle the text part. They take a long prompt describing a scene and split it into two parts: who is in the picture, and what the setting looks like. Then they inject a tiny appearance correction—a residual—into the scene's main embedding.

Tom: That residual is key because it’s supposed to carry that phrase-level appearance information into the scene context, so you don't lose details when you make your prompt really long.

Jane: And Adaptive Identity Preservation is for the picture itself. They deal with structural variance, which means making sure the subject looks right in a new frame or at a different scale than before.

Tom: They achieve this by doing Identity Anchoring, which basically resizes the identity features to match how big the scene is supposed to be, enforcing structural compatibility between what you want and what’s actually there.

Jane: Then they use Identity-Aware Self-Attention to make sure the scene features are actually looking at those resized identity features while generating.

Tom: And they tested this on an extended benchmark called Consistory++, which has a ton of different framing scenarios, and it performs better than existing methods in both prompt accuracy and structural adaptability.

Jane: It seems like the real win here is that this method keeps the visual fidelity high—the aesthetic scores are better—while also being much more flexible when you’re trying to change the scene context.

Tom: The authors point out that they showed a residual injection at the EOS token works as a subject-agnostic direction, which means it’s transferable across different subjects too.

Jane: It changes things for users because if they are using these models for storytelling or visual consistency, they don't have to rewrite their entire setup every time the environment shifts.

Tom: It also addresses how people actually use these things by including a prompt parser with GPT-4o, so messy human instructions can still be fed into the system correctly.

Jane: So, in short, EOS-Story is a training-free way to get better scene storytelling and visual consistency across different contexts.

Lu: The core contribution is showing that the EOS embedding can carry phrase-level, multi-attribute information, enabling appearance cues to be infused during generation <ref:2610.22320#pg3>.

Meng: But they did flag a limitation—they mentioned that existing approaches still struggle with semantic misalignment in simple cases or losing fine-grained prompt details in complex ones <ref:2610.22320#pg1>.

The paper's summary: Tom: So we’re talking about how they suggest improving their framework, EOS-Story, which is really about making those core mechanisms even sharper.

Jane: What do they suggest? I mean, the paper shows a lot of stuff already working well on Consistory++ and other benchmarks.

Tom: They focus on the ablation studies where Cross-Contextual Bridging was shown to improve CLIP-T scores, which is good because it means their method for injecting appearance residuals into the EOS embedding is definitely helping with text alignment <ref:2610.22320#pg3>.

Jane: So they’re saying that focusing on that specific residual injection at the EOS token is what makes the subject appearance cues transferable and controllable, which sounds like a really useful finding for general use.

Tom: Right, because they showed that this residual functions as a direction that doesn't care about the subject itself, making it very flexible <ref:2610.22320#pg3>.

Jane: And on the structural side, they highlighted that Adaptive Identity Preservation contributes more to those identity-related scores like CLIP-I than just the basic anchoring part.

Tom: So they’re telling us that resizing features to match scene scale is a bigger lever for getting the subject features right than just using the basic correspondence map <ref:2610.22320#pg3>.

Jane: That implies that if you want better fidelity in a specific frame, focusing your tuning on how identity features match the scene's size will yield more results.

Tom: They also mentioned that Dynamic Switching Timestep in Adaptive Identity Preservation is important because it lets you balance prompt alignment and identity consistency by transitioning at t equals nine <ref:2610.22320#pg3>.

Jane: That means the model can dynamically switch its focus, balancing prompt fidelity against identity preservation depending on what’s happening in the generation process.

Tom: It’s about giving the model a knob to turn between those two goals, so you don't have to pick just one setting for everything.

Jane: What this changes for someone who uses this is that they get more control over the output; they can fine-tune whether they prioritize prompt accuracy or subject consistency based on what they are trying to achieve in a specific scene.

Tom: And Lalam confirms that the authors are pretty sure these improvements are solid, especially since it works across those extended benchmarks like Consistory++ <ref:2610.22320#pg1>.

Jane: So, the implication is that for anyone building on this, they shouldn't just focus on the main framework; they should look at tuning those specific components—the residual injection and the dynamic switching—for maximum effect.

Lu: It’s about refining the details of how those semantic controls work to get better results in a real-world scenario.

The paper's improvements: Tom: So we’re wrapping up on "Not Just a Subject: Capturing Adaptive Scene-Specific Variance for Visual Storytelling" by Cha et al., which is basically their framework for making AI images handle scene changes better without retraining.

Jane: It’s about taking that subject consistency and pairing it with scene adaptation using those two main methods we talked about, Cross-Contextual Bridging and Adaptive Identity Preservation.

Tom: Exactly, the implication is that you can get better text alignment without compromising image quality metrics like Aesthetic scores or ImageReward <ref:2610.22320#pg1>.

Jane: And for users who only listen to the show, it means they can use these models for visual storytelling where they need consistency across different settings, and they don't have to do all that manual prompt engineering.

Tom: Lalam says I’m pretty sure this is solid because the authors showed consistent improvements over existing approaches on both the original and extended benchmarks like Consistory++ <ref:2610.22320#pg1>.

Jane: Right, and we saw how it performed well in those complex framing scenarios, which shows it handles scene variance quite well under varied viewpoints.

Tom: It’s a training-free method designed to harmonize identity consistency with scene-specific adaptation across text and vision spaces <ref:2610.22320#pg1>.

Jane: I think the real win is seeing that residual appearance injection at the EOS token acts as a subject-agnostic direction, which means it’s transferable across different subjects too.

Tom: That’s a clean finding because it shows a mechanism for controlling appearance that doesn't rely on knowing what the specific subject is.

Jane: It really solidifies that these residual controls are just how the model learns to reflect the intended semantics during generation, which is pretty smart design.

Tom: So, basically, they’ve given us a robust way to handle scene variance using structural and semantic control points in a way that doesn't require massive datasets.

Jane: It’s a solid piece of research for anyone building visual agents or storytelling systems that need to be flexible in their environments.

Tom: For Meng, what about the deployment side? Does this framework introduce any weird computational costs when you try to run it at scale?

Meng: I'm worried about those complex attention mechanisms, but since it's training-free, the inference cost seems comparable to existing methods for generating five stories <ref:2610.22320#pg3>.

Jane: That’s good because if the cost is comparable, it makes deployment much more feasible for production systems that need consistent output quickly.

Tom: Lu thinks the approach of anchoring features at scene-specific scales is a really strong concept because it directly addresses how structural distortions happen in visual generation <ref:2610.22320#pg3>.

Jane: So, the idea is that by matching the identity feature size to the scene's scale, you enforce structural coherence and keep things looking right.

Tom: And Lalam confirms that this works reliably because they found that size matching promotes structural coherence and improves image quality metrics significantly <ref:2610.22320#pg3>.

Jane: It’s interesting how they tie the visual fidelity directly into a structural constraint; it makes sense that the subject features would behave more predictably under those conditions.

Tom: We’re done with this paper, "Not Just a Subject: Capturing Adaptive Scene-Specific Variance for Visual Storytelling."

Jane: It’s been really interesting seeing how they managed to harmonize identity consistency with scene-specific adaptation using these training-free techniques.

Lu: I think the real win is seeing that residual appearance injection at the EOS token acts as a subject-agnostic direction, which means it’s transferable across different subjects too <ref:2610.22320#pg3>.

Meng: For deployment, the key seems to be that it maintains image quality metrics while being flexible enough for real-world user inputs <ref:2610.22320#pg1>.

Tom: Next up, we’re going to look at the work on KiToke, which tackles token compression for video large language models.

Conclusion: Tom: So we're wrapping up on "Not Just a Subject: Capturing Adaptive Scene-Specific Variance for Visual Storytelling" by SeungJu Cha et al., which is basically their framework for making AI images handle scene changes without retraining.

Jane: It’s about taking that subject consistency and pairing it with scene adaptation using those two main methods we talked about, Cross-Contextual Bridging and Adaptive Identity Preservation.

Tom: Exactly, the implication is that you can get better text alignment without compromising image quality metrics like Aesthetic scores or ImageReward.

Tom: Lalam says I’m pretty sure this is solid because the authors showed consistent improvements over existing approaches on both the original and extended benchmarks like Consistory+.

Jane: Right, and we saw how it performed well in those complex framing scenarios, which shows it handles scene variance quite well under varied viewpoints.

Tom: It’s a training-free method designed to harmonize identity consistency with scene-specific adaptation across text and vision spaces.

Jane: I think the real win is seeing that residual appearance injection at the EOS token acts as a subject-agnostic direction, which means it’s transferable across different subjects too.

Tom: That’s a clean finding because it shows a mechanism for controlling appearance that doesn't rely on knowing what the specific subject is.

Jane: It really solidifies that these residual controls are just how the model learns to reflect the intended semantics during generation, which is pretty smart design.

Tom: So, basically, they’ve given us a robust way to handle scene variance using structural and semantic control points in a way that doesn't require massive datasets.

Jane: It’s a solid piece of research for anyone building visual agents or storytelling systems that need to be flexible in their environments.

Tom: For Meng, what about the deployment side? Does this framework introduce any weird computational costs when you try to run it at scale?

Meng: I'm worried about those complex attention mechanisms, but since it's training-free, the inference cost seems comparable to existing methods for generating five stories.

Jane: That’s good because if the cost is comparable, it makes deployment much more feasible for production systems that need consistent output quickly.

Tom: Lu thinks the approach of anchoring features at scene-specific scales is a really strong concept because it directly addresses how structural distortions happen in visual generation.

Jane: So, the idea is that by matching the identity feature size to the scene's scale, you enforce structural coherence and keep things looking right.

Tom: And Lalam confirms that this works reliably because they found that size matching promotes structural coherence and improves image quality metrics significantly.

Jane: It’s interesting how they tie the visual fidelity directly into a structural constraint; it makes sense that the subject features would behave more predictably under those conditions.

Tom: We’re done with this paper, "Not Just a Subject: Capturing Adaptive Scene-Specific Variance for Visual Storytelling."

Jane: It’s been really interesting seeing how they managed to harmonize identity consistency with scene-specific adaptation using these training-free techniques.

Tom: Next up, we're going to look at the work on KiToke, which tackles token compression for video large language models.

SeungJu Cha, Ye-Chan Kim, Kwanyoung Lee, Dong-Jin Kim

Hanyang University

cs.CV

Submitted: 2026-10-15

Updated: 2026-10-15

Comments: Accepted at ACCV 2026. Code: https://github.com/SeungJuCha/EOS-Story

Code: https://github.com/SeungJuCha/EOS-Story

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 90/100

The gist: The gist: EOS-Story is a training-free framework that harmonizes identity consistency with scene-specific adaptation across text and vision spaces.

Key concepts

Cross-Contextual Bridging (CCB): This module separates the narrative into an identity prompt (describing the subject) and a scene prompt (describing the setting). It then uses an <EOS> embedding to inject phrase-level appearance cues from the identity into the scene description. This transfers visual details without making text prompts unnecessarily long, ensuring appearance information is present in every scene context.
Adaptive Identity Preservation (AIP): This mechanism handles structural differences by resizing subject features to match the specific scale of each scene. It enforces structural compatibility between the identity and the scene representations. This ensures that even when scenes have different framing or scales, the core identity features are correctly aligned before being integrated into the final output.
Identity Anchoring (IA): A component of AIP that adaptively resizes identity features to align with the intended framing and scale of a particular scene. By computing a correspondence map between size-aligned identity features and scene subject features, it identifies the most similar position for each frame. This reduces distortions caused by mismatched scales.
Identity-Aware Self-Attention (IASA): This refinement step allows each scene's self-attention mechanism to dynamically focus on relevant identity features. It achieves this by concatenating key and value embeddings from the identity image, enabling scenes to selectively attend to subject details during generation, leading to better integration of identity into the generated image.

Terminology

Benchmark — 2× in this paper · explained in 2 episodes
Adaptation — 1× in this paper · explained in 1 episode
Consistency — 1× in this paper · explained in 3 episodes
Diffusion Model — 1× in this paper · explained in 10 episodes
Diffusion Models — 1× in this paper · explained in 9 episodes
Training-Free Framework — 1× in this paper · explained in 2 episodes

Summary

The gist: EOS-Story is a training-free framework that harmonizes identity consistency with scene-specific adaptation across text and vision spaces.

How it works

Cross-Contextual Bridging (CCB) addresses semantic variance by transferring phrase-level appearance cues into the scene context through the ⟨EOS⟩ embedding, mitigating information dilution in long and complex prompts. This involves decoupling the narrative into an identity prompt describing the subject and a scene prompt describing the scenario, and then softly injecting appearance information at the embedding level via a residual-based semantic control point for scene-aware storytelling. The appearance residual is defined as the local displacement from the shared style-subject anchor to the identity prompt, following CLIP embedding space linearity. This results in an infused embedding eosinf = eossc + Δapp, which restores subject appearance cues within the scene prompt representation while preserving original scene semantics.
Adaptive Identity Preservation (AIP) addresses structural variance by aligning identity features to each scene’s subject scale (Identity Anchoring), followed by Identity-Aware Self-Attention refinement. Identity Anchoring adaptively resizes identity features to match the intended framing and scale of the scene, enforcing structural compatibility between identity and scene representations. This is achieved by computing a correspondence map between the size-aligned identity feature x˜n id and the scene subject feature xˆn sc, identifying the most similar position for each scene position i ∈ [1, hw]. The blended scene subject and matched identity features are then computed using a weighting coefficient λ to balance scene context preservation and identity transfer. Identity-Aware Self-Attention refines this integration by enabling each scene to dynamically attend to identity features during self-attention, building upon prior work.

Evaluation and Contributions

The framework is evaluated under varied spatial compositions using the extended Consistory+ benchmark, which includes diverse framing scenarios. Extensive experiments show that EOS-Story achieves competitive or superior performance on both benchmarks. The contributions include showing that the ⟨EOS⟩ embedding can carry phrase-level, multi-attribute information, enabling appearance cues to be infused during generation. It also introduces Consistory++, an extended benchmark with diverse framing phrases, and demonstrates consistent improvements over existing approaches on both the original and extended benchmarks.

Key Findings

** EOS-Story achieves better text alignment without compromising image quality, as shown in Table 1. It also shows substantial improvements in image quality metrics (Aesthetic, ImageReward), confirming its ability to maintain visual fidelity. For framing accuracy (Fig. 5), EOS-Story also achieves the best results on Consistory++, which evaluates whether the framing phrases are properly reflected. Qualitative results show that our model faithfully reflects the given scenario while showing a flexible subject in each image, and it successfully captures the intended framing while preserving both prompt and subject fidelity across scenes. The analysis of EOS linearity suggests that residual appearance injection at the EOS token helps effectively reflect the intended semantics during generation. This residual functions as a subject-agnostic, transferable, and controllable appearance direction. The ablation studies confirm that Cross-Contextual Bridging improves CLIP-T, while Adaptive Identity Preservation contributes more prominently to identity-related scores such as CLIP-I. Size matching promotes structural coherence and improves image quality. The user study indicates that EOS-Story was selected as the best in 52.8% of cases when balancing identity, prompt alignment, and scene adaptability across 20 participants. The cost of generating 5 stories is comparable to existing methods. This work demonstrates that EOS-Story effectively adapts to scene variance, ultimately producing enhanced visual storytelling outputs >.">Page 1

The gist: EOS-Story is a training-free framework that harmonizes identity consistency with scene-specific adaptation across text and vision spaces.">Page 1

This paper proposes EOS-Story, a training-free framework that adaptively models scene-specific variance in visual storytelling across text and vision spaces. We introduce Cross-Contextual Bridging, which decouples the narrative into identity and scene sub-prompts and bridges the resulting semantic gap by injecting phrase-level appearance residuals into the scene’s ⟨EOS⟩ embedding. To complement this with structural adaptation, we further propose Adaptive Identity Preservation, which anchors subject features at the scene-specific scale and refines integration through Identity-Aware Self-Attention. To evaluate scene adaptability, we extend Consistory+ with diverse framing variations (Consistory++). Extensive experiments show that EOS-Story achieves competitive or superior performance in both prompt fidelity and structural adaptability.">Page 1

Cross-Contextual Bridging (CCB) addresses semantic variance by transferring phrase-level appearance cues into the scene context through the ⟨EOS⟩ embedding, mitigating information dilution in long and complex prompts.

Improvements for AI systems

Bold header: Cross-Contextual Bridging (CCB) for Semantic Recomposition. This module decouples the narrative into an identity prompt describing the subject and a scene prompt describing the scenario and softly injects a phrase-level appearance residual into the scene’s ⟨EOS⟩ embedding, allowing for phrase-level appearance transfer without increasing the textual length of the scene description.
Bold header: Adaptive Identity Preservation (AIP) via Identity Anchoring (IA). This mechanism addresses structural variance by applying Identity Anchoring, which adaptively resizes identity features to match the intended framing and scale of the scene, ensuring that scale-induced distortions are reduced by enforcing structural compatibility between identity and scene representations.
Bold header: Identity-Aware Self-Attention (IASA) for Spatially Adaptive Transfer. This refinement enables scene features to selectively attend to subject features by concatenating key and value embeddings from the identity image, leading to a mechanism that can dynamically attend to identity features during self-attention.
Bold header: Dynamic Switching Timestep in AIP. The framework allows for balancing prompt alignment and identity consistency by transitioning at t = 9 effectively balances prompt alignment (CLIP-T) and identity consistency (CLIP-I), ensuring both structural flexibility and identity preservation.
Bold header: Robust Evaluation via Extended Benchmarks. The system can be rigorously tested on complex visual scenarios using Consistory++, an extended benchmark with diverse framing phrases, allowing it to adaptively capture scene variances under varied viewpoints and scales, as demonstrated by superior performance in the evaluation metrics.
Bold header: Free-form Prompt Parsing via GPT-4o. The model can handle unstructured user inputs by employing GPT-4o as a prompt parser that converts free-form user instructions into the intended prompt format for EOS-Story, supporting more flexible user inputs.