Mixture of Multi-Sequence Feature Difference for Contrast-Enhanced MRI Synthesis

arXiv:2610.21703 · cs.CV · Submitted 2026-10-15 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "Mixture of Multi-Sequence Feature Difference for Contrast-Enhanced MRI Synthesis".

Jane: The gist The proposed method introduces a Mixture of Feature Difference (MoFD) module to synthesize contrast-enhanced magnetic resonance imaging (CE-MRI) from non-contrast sequences by adaptively selecting and aggregating informative features…

Tom: First, who's behind it and why it matters.

Paper summary: Jane: So, to wrap up this paper on "Mixture of Multi-Sequence Feature Difference for Contrast-Enhanced MRI Synthesis", they successfully synthesized T1ce images from non-contrast multisequence MRI by explicitly modeling both sequence-specific features and their pairwise feature differences.

Tom: The main thing is that the Mixture of Feature Difference module lets the network adaptively select and aggregate informative features from both individual source sequences and explicit differences among them to produce a final feature representation suitable for target T1ce synthesis.

Lu: What this means practically is that instead of just concatenating channels, you’re modeling how things change in contrast across different modalities, which gives the network a much richer context for synthesizing the final image.

Meng: For someone who runs this model to make a real application, it suggests that explicitly modeling those differences is crucial if you want to preserve tumor structure while getting the contrast effect right.

Lalam: They proved that their approach consistently outperformed baseline methods in image fidelity and improved downstream tumor segmentation performance on the UCSF-PDGM and UPenn-GBM datasets.

Jane: The authors highlighted that regression-based training can actually preserve structural fidelity better than adversarial training when you really need accurate representation of the tumor.

Tom: So, the title points to this method being about using those feature differences to synthesize those contrast-enhanced images, and it seems like this explicit modeling of inter-sequence variations is a solid way to reduce risk and cost compared to traditional methods.

Conclusion: Tom: So we've looked at how this Mixture of Multi-Sequence Feature Difference method actually works for synthesizing contrast-enhanced MRI images from non-contrast sequences.

Jane: Yeah, so the authors are trying to make those T1ce images without having to inject any actual contrast agent, which is a big deal because of the safety risks and the high scan costs we talked about earlier.

Tom: Right. The title itself, "Mixture of Multi-Sequence Feature Difference," it’s basically telling us they aren't just mixing up different MRI scans randomly; they are specifically looking at how those sequences differ from each other to figure out what the contrast should look like.

Lu: Exactly. They are extracting features from each sequence individually, and then they calculate these explicit feature differences between them, which forms the core of their Mixture of Feature Difference module.

Jane: So, instead of just feeding all the data into one big bucket, this module lets them dynamically decide which pieces of information—the individual sequence features or the differences between them—are most helpful for building the final contrast image.

Tom: That dynamic weighting strategy is what really gets me. It’s not a fixed recipe; it adapts based on what’s useful for that specific synthesis task, and they're seeing better results on those UCSF and UPenn datasets.

Lu: And I'm interested because the ablation studies showed that even though adding an adversarial loss could make the images look prettier, a simpler regression-based version actually kept the structural details of the tumor much better.

Jane: That’s a crucial detail for clinicians because if you want to see where a tumor is precisely, you need that structural fidelity, not just something that looks visually appealing in one specific way.

Tom: So what this means for us is that we might be able to create these useful contrast-enhanced images using only standard non-contrast sequences without the hassle or the risk of gadolinium injection.

Lu: It suggests a new way to leverage existing MRI data, focusing on the relationships between those images rather than just treating them as separate inputs.

Jane: That really points toward a future where we could get better tumor characterization without increasing patient exposure to contrast agents.

Jonghun Kim, Junmo Kwon, Hyunjin Park

Department of Electrical and Computer Engineering, Sungkyunkwan University

cs.CV

Submitted: 2026-10-15

Updated: 2026-10-15

Comments: ACCV 2026

Code: https://github.com/jongdory/MoFD

License: http://creativecommons.org/licenses/by-nc-sa/4.0/

Importance score: 82/100

The gist: The gist The proposed method introduces a Mixture of Feature Difference (MoFD) module to synthesize contrast-enhanced magnetic resonance imaging (CE-MRI) from non-contrast sequences by adaptively

Key concepts

Mixture of Feature Difference (MoFD) Module: This is the core component that fuses information from multiple MRI sequences. Instead of simply stacking data, it extracts features from each sequence and also calculates the differences between these features. A dynamic weighting strategy then decides how much to rely on each individual feature versus these calculated differences to create a refined representation for synthesis.
Reconstruction Loss (L_rec): This loss function measures how accurately the synthesized image matches the true ground-truth CE-MRI image. It uses an L1 norm, which penalizes large errors in pixel values. Minimizing this loss ensures that the final synthesized image looks visually and quantitatively like a real contrast-enhanced scan.
Structural Alignment Loss (L_align): This loss function focuses on ensuring that the underlying anatomy is consistent across different input sequences. It uses a structural similarity index measure (SSIM) to compare features from different sequences. This forces the model to maintain correct anatomical relationships while synthesizing contrast information.
Adversarial Loss (L_gan): This optional loss term is used when the goal is improved perceptual quality rather than just structural accuracy. It employs a Generative Adversarial Network (GAN) setup to train the model to produce outputs that look more realistic, enhancing the visual quality of the synthesized CE-MRI image.

Terminology

Fidelity — 2× in this paper · explained in 2 episodes
Ablation — 1× in this paper · explained in 1 episode
Ablation Studies — 1× in this paper · explained in 1 episode
Dynamic Weighting — 1× in this paper · explained in 3 episodes

Summary

The gist The proposed method introduces a Mixture of Feature Difference (MoFD) module to synthesize contrast-enhanced magnetic resonance imaging (CE-MRI) from non-contrast sequences by adaptively selecting and aggregating informative features from both individual source sequences and explicit differences among them.

Motivation

Contrast-enhanced magnetic resonance imaging (CE-MRI) is crucial for accurately diagnosing and segmenting brain tumors, but the acquisition of CE-MRI requires injecting contrast agents, which poses potential health risks and increases scan costs <ref:2610.21703#pg16>. Non-contrast MRI often lacks the sensitivity needed to identify vascular conditions and subtle tissue discrepancies because conditions like tumors or inflammation can have similar intensities on non-contrast scans <ref:2610.21703#pg5>. CE-MRI addresses this by using contrast agents that make blood vessels and lesions hyperintense on T1-weighted images, which enables the precise identification of the location and extent of these lesions <ref:2610.21703#pg6>. Despite its advantages, administering contrast agents raises safety concerns because gadolinium may not be fully excreted, potentially leading to accumulation in the brain and bones <ref:2610.21703#pg5>. Furthermore, CE-MRI requires longer acquisition times and incurs higher costs <ref:2610.21703#pg6>.

Proposed Method (MoFD Module)

The MoFD module is introduced to effectively leverage the complementary information from multiple sequences for CE-MRI synthesis <ref:2610.21703#pg4>. Unlike conventional methods that concatenate sequences across channels, the proposed approach uses an encoder to extract sequence-specific features from each input sequence capturing unique contrast characteristics <ref:2610.21703#pg6>. The process involves several key steps:

Extracting multi-scale features from each input sequence I k using a weight-sharing encoder <ref:2610.21703#pg6>.
Defining the feature difference set D i at the i-th scale as the collection of pairwise differences between sequence features, defined as D i =

<ref:2610.217035>.

Forming a combined representation C i by concatenating individual features Fi and feature difference set Di, and then processing it through a convolution block to yield the refined feature map F i <ref:2610.21703#pg6>.
Computing attention weights ai by applying Global Average Pooling (GAP) followed by a softmax function <ref:2610.21703#pg6>.
Generating the final aggregated feature ¯fi as a weighted sum of these features: ¯f i = sum from m=1 to M of a m i.tilde f m i <ref:2610.21703#pg6>.
Passing the aggregated multi-scale features F to the decoder to synthesize the CE-MRI, defined as ˆY = Decoder(¯F) <ref:2610.21703#pg6>.

Training Strategy and Loss Functions

The model is optimized using a total objective function formulated as L total = L rec + L align <ref:2610.21703#pg6>.

** Reconstruction Loss (L rec)**

The reconstruction loss encourages the predicted image Y to closely match the ground truth Y, defined as ˆY = Decoder(Y) and L rec = Y - ˆY 1 <ref:2610.21703#pg6>.

** Structural Alignment Loss (L align)**

The structural alignment loss enforces structural consistency across different sequences for a single subject, defined as L align = 1 - s(f k i, f l i) <ref:2610.21703#pg6>. This loss is computed by focusing on the structure comparison component of the Structural Similarity Index Measure (SSIM) [50] for the sequence feature f k i <ref:2610.21703#pg6>.

** Adversarial Loss (L gan)**

An optional adversarial loss, L gan = E Y[log D(Y)] + E ˆY[log (1 - D(G(S))] is used when improved perceptual quality is desired <ref:2610.21703#pg6>. The total objective function with the GAN term is formulated as L total = L rec + L align + L gan <ref:2610.21703#pg6>.

Experimental Validation and Results

The method was extensively validated on the UCSF-PDGM and UPenn-GBM datasets, which consist of multi-sequence MRI including T1, T2, T2-FLAIR, and T1ce images <ref:2610.21703#pg5>. Quantitative evaluations using 3D PSNR and SSIM metrics demonstrated that the regression-based method (MoFD w/o GAN) achieved the best fidelity performance <ref:2610.21703#pg6>. Qualitative comparisons showed that while several comparison methods fail to capture detailed tumor appearance, the proposed method reconstructs both anatomical structures and contrast-enhanced tumor regions more faithfully <ref:2610.21703#pg6>. Furthermore, ablation studies indicated that feature normalization and dynamic weighting are important for stable fusion, all input sequences and their difference features are beneficial for synthesis quality, and regression-based training can better preserve structural fidelity than adversarial training when accurate tumor representation is required <ref:2610.21703#pg6>. The Top-K feature selection strategy showed that the best performance was achieved when all six available features, comprising three sequence features and three difference maps, across both datasets was better than selecting a partial subset <ref:2610.21703#pg6>.

Conclusion

The framework successfully synthesizes T1ce images from non-contrast multisequence MRI by explicitly modeling both sequence-specific features and their pairwise feature differences, allowing the network to adaptively select complementary information useful for contrast-enhanced image synthesis <ref:2610.21703#pg6>. The experiments consistently outperformed baseline approaches in image fidelity and improved downstream tumor segmentation performance <ref:2610.21703#pg6>. The ablation studies further demonstrated the contribution of the proposed design, confirming that feature normalization and dynamic weighting are important for stable fusion, all input sequences and their difference features are beneficial for synthesis quality, and regression-based training can better preserve structural fidelity than adversarial training when accurate tumor representation is required <ref:2610.21703#pg6>. The MoFD module effectively extracts complementary representations and dynamically integrates them to produce a final feature representation suitable for target T1ce synthesis <ref:2610.21703#pg6>.

Acknowledgements

This study was supported by the SMC-SKKU Future Convergence Research Program Grant (SMO125123) <ref:2610.21703#pg6>. It was also supported by the National Research Foundation of Korea (NRF) grant (RS-2025-00518533), the AI Graduate School Support Program (Sungkyunkwan University) (RS-2019-II190421), the National Research Foundation of Korea (RS-2024-00408040, RS-26598771), the ICT Creative Consilience Program Grant (IITP-26598771), the Institute of Information & Communications Technology Planning & Evaluation (IITP) grant (RS-26598771, Resource-Intensive AI Technologies Based on Sustainable GPU Integrated Platforms), the IITP grant (RS-26598771, High-Performance Research AI Computing Infrastructure Support at the 2 PFLOPS Scale), all funded by the Ministry of Science and ICT (MSIT), Republic of Korea <ref:2610.21703#pg6>.

--- Page 1 ---

Motivation

Contrast-enhanced magnetic resonance imaging (CE-MRI) is crucial for accurately diagnosing and segmenting brain tumors, but the acquisition of CE-MRI requires injecting contrast agents, which poses potential health risks and increases scan costs <ref:2610.21703#pg6>. Non-contrast MRI often lacks the sensitivity needed to identify vascular conditions and subtle tissue discrepancies because conditions like tumors or inflammation can have similar intensities on non-contrast scans <ref:2610.21703#pg5>. CE-MRI addresses this by using contrast agents that make blood vessels and lesions hyperintense on T1-weighted images, which enables the precise identification of the location and extent of these lesions <ref:2610.21703#pg6>. Despite its advantages, administering contrast agents raises safety concerns because gadolinium may not be fully excreted, potentially leading to accumulation in the brain and bones <ref:2610.21703#pg5>.

Improvements for AI systems

Dynamic Feature Selection via MoFD Module Enhancement: The improved system can adaptively select and aggregate informative features by using a dynamic weighting strategy to adaptively select and aggregate informative features from both individual source sequences and explicit differences among them. This allows for flexible and selective feature integration instead of fixed averaging, leading to a refined representation that emphasizes relevant cross-modality cues while reducing the influence of redundant or less informative features.
Structural Consistency Enforcement: The system can be enhanced by incorporating the structural alignment loss defined as s(f k i, f l i) = (σ(f k i, f l i)+ ε) / (σ(f k i) σ(f l i) + ε) to enforce structural consistency across multiple sequences for a single subject, which helps the model preserve patient-specific structure during synthesis.
Loss Function Optimization: The system can be optimized by prioritizing structural fidelity over perceptual quality by using a regression-based objective, as the paper states, regression-based variant of our method achieves higher structural fidelity compared to adversarial learning when accuracy is critical. Furthermore, incorporating the alignment loss improved both fidelity and quality metrics in all cases.
Feature Selection Granularity: The Top-K selection strategy can be leveraged to determine feature importance by varying K, with results showing that all six available features, comprising three sequence features and three difference maps, yield the best performance across both datasets. This allows the model to effectively exploit complementary representations by using all available cues.

Abstract

Contrast-enhanced magnetic resonance imaging (CE-MRI) is crucial for accurately diagnosing and segmenting brain tumors. However, the acquisition of CE-MRI requires injecting contrast agents, which poses potential health risks and increases scan costs. Here, we propose a CE-MRI synthesis method from non-contrast sequences, reducing both risk and cost. Unlike existing methods that rely on channel-wise concatenation to fuse multi-sequence information for synthesis, we introduce a Mixture of Feature Difference (MoFD) module. The proposed module uses a dynamic weighting strategy to adaptively select and aggregate informative features from both individual source sequences and explicit differences among them. Quantitative and qualitative evaluations demonstrate that our proposed method outperforms baselines in terms of image fidelity and perceptual quality on the UCSF-PDGM and UPenn-GBM datasets. Our method successfully reconstructs tumor regions and achieves segmentation performance comparable to that obtained with real CE-MRI. Our ablation studies show that although adversarial learning improves perceptual quality, a regression-based variant of our method achieves higher structural fidelity.