Registration-Grounded Spectral Fusion for Unregistered WLI/NBI Endoscopic Lesion Segmentation

arXiv:2607.26395 · cs.CV · Submitted 2026-07-29 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "Registration-Grounded Spectral Fusion for Unregistered WLI/NBI Endoscopic Lesion Segmentation".

Jane: Here's the gist of the paper. The gist The proposed framework addresses paired-but-unregistered WLI/NBI lesion segmentation by coupling topology-regularized correspondence grounding with reliability-guided complex-domain interaction to exploit modality complementarity while suppressing unreliable cross-modal interaction.

Tom: Problem and Motivation White-light imaging (WLI) and narrow-band imaging (NBI) provide complementary views of endoscopic lesions, but their paired observations are often spatially misaligned due to viewpoint changes, tissue deformation, and sequential handheld acquisition <ref:2607.26395#pg2> The lack of task-relevant pixel-topixel correspondence makes direct WLI/NBI fusion prone to mixing non-corresponding regions and may even degrade segmentation around lesion boundaries <ref:2607.26395#pg2> This difficulty highlights a gap between clinical dual-modality observation and current automatic segmentation practice <ref:2607.26395#pg2> Existing methods mainly improve feature compatibility or introduce coarse alignment, but they do not explicitly establish topology-regularized feature correspondence together with local correspondence reliability before cross-modal interaction <ref:2607.26395#pg2> Furthermore, most WLI/NBI fusion methods aggregate features in the spatial or real-valued domain and often treat WLI and NBI as exchangeable information sources <ref:2607.26395#pg2> Therefore, effective WLI/NBI segmentation requires reliability-aware correspondence modeling and role-aware complementary fusion <ref:2607.26395#pg2>.

Jane: Proposed Framework Components The framework consists of a modality-aware pyramid encoding, Reliability-Aware Diffeomorphic Registration Grounding (R-DRG), Reliability-Guided Complex Spectral Fusion (R-CSF), and dual-branch segmentation decoding <ref:2607.26395#pg2>.

Tom: 1.

Title and authors: Tom: We’re diving into this paper titled "Registration-Grounded Spectral Fusion for Unregistered WLI/NBI Endoscopic Lesion Segmentation" and who wrote it. It’s authored by Pengyu Jiea, Wanquan Liua, Rui Heb, Pengcheng Lic, Weiping Wenb, Deyu Mengd, Junwei Hane and Chenqiang Gaoa.

Jane: That’s a pretty long list of authors for one paper that tackles this specific medical imaging problem. What does the title tell us about the main hurdle they’re trying to clear?

Lu: The title itself tells us everything. It says "Registration-Grounded Spectral Fusion for Unregistered WLI/NBI Endoscopic Lesion Segmentation". The key hurdles are unregistered views and the need for registration grounded fusion.

Meng: So, let's break down what that means in plain terms. "Unregistered" means we don't have perfectly paired images from the start, because they might be spatially misaligned due to things like tissue deformation or different acquisition times.

Tom: Right. And "Registration-Grounded" is the mechanism they use to solve that misalignment problem before they try to fuse anything together. It means the fusion process isn't blind; it’s guided by a proper geometric alignment first.

Jane: So, if we had just tried to fuse them directly without any alignment, what would be the problem?

Lu: The paper says that direct WLI/NBI fusion is prone to mixing non-corresponding regions and might even degrade the segmentation around lesion boundaries because of those spatial mismatches.

Meng: That’s exactly what we see in practice. If you mix mismatched areas, your segmentation map gets garbage, especially around the important parts like lesion edges.

Tom: And this paper sets out to fix that by making sure the correspondence is reliable before they even get to the fusion stage itself. It’s about establishing that trust first.

Jane: So it sounds like they are tackling two big problems at once: how to align mismatched images and how to fuse them intelligently when you can't fully trust the alignment.

Lu: Precisely, they address both the alignment issue and the intelligent fusion issue simultaneously through their framework.

The paper's summary: Tom: So let’s talk about the paper’s summary of "Registration-Grounded Spectral Fusion for Unregistered WLI/NBI Endoscopic Lesion Segmentation". It outlines their overall approach.

Jane: Can you explain that to me simply, like I’m just listening to the high-level idea? What are the core components they propose?

Lu: The framework consists of a modality-aware shared encoder, ReliabilityGuided Diffeomorphic Registration Grounding or R-DRG, Reliability-Guided Complex Spectral Fusion or R-CSF, and dual-branch segmentation decoders.

Meng: That sounds like a lot of moving parts. Can you walk me through what each part is supposed to do?

Tom: First is the encoder, which’s modality aware. Then we have R-DRG, which establishes topology-preserving correspondence and derives reliability maps.

Jane: And R-CSF uses those maps to guide the fusion of WLI appearance cues and NBI structural cues selectively based on local reliability.

Lu: So, R-CSF is where they use those maps to guide role-aware complex spectral fusion, which means WLI cues parameterize magnitude responses and NBI cues parameterize phase responses.

Meng: That’s the key idea—they are not treating the two modalities as interchangeable information sources in a spatial or real-valued domain.

Tom: Correct. And R-CSF then uses those maps to decide how strongly those cross-modal spectral cues should be injected, allowing them to fall back to modality-specific cues when correspondence is unreliable.

Jane: So it’s a smart way to handle the uncertainty; you use the maps as priors for fusion rather than just treating them like any other data source.

Lu: The joint optimization includes segmentation loss, smoothness regularization on the velocity field, and probability-level constraints comparing probability masks after geometric transformation.

Meng: So it’s not just one big loss function; they are using multiple constraints to keep the geometry smooth and the probabilities consistent across both modalities.

The paper's improvements: Tom: Now let’s get into what makes this method better than what came before. What specific improvements are they suggesting?

Jane: The main improvement seems to be moving beyond just feature alignment or coarse registration, to explicitly modeling topology-regularized feature correspondence together with local correspondence reliability before cross-modal interaction.

Lu: Yes, that’s the first big one: preventing residual local mismatch from causing unreliable feature fusion around lesion boundaries and fine mucosal structures.

Meng: That directly addresses the problem of mixing non-corresponding regions and ruining boundary segmentation quality.

Tom: And secondly, they improve the fusion strategy by using reliability maps to control spectral intensity and angular response via that equation in (eight).

Jane: So, high reliability regions get stronger spectral recalibration, while low-reliability regions preserve more of the original modality-specific complex feature.

Lu: And third is topology-preserving feature mapping, where they utilize bidirectional diffeomorphic transformations derived from a single stationary velocity field to provide a smooth and topologypreserving geometric basis before cross-modal fusion.

Meng: That smooth geometric basis should make the fusion process much more robust to viewpoint changes, tissue deformation, and sequential handheld acquisition.

Tom: So in short, they’ve improved the framework by focusing on reliability first, then using topology to align smoothly before applying a role-aware complex spectral fusion.

Conclusion: Jane: We’ve covered the mechanics of this paper called "Registration-Grounded Spectral Fusion for Unregistered WLI/NBI Endoscopic Lesion Segmentation" and what its core improvements are. Where does it all lead us now?

Lu: It provides a principled fusion paradigm for paired-but-unregistered WLI/NBI segmentation by coupling topology-regularized correspondence grounding with role-aware complex spectral fusion, which effectively reduces negative transfer from locally mismatched WLI/NBI observations.

Meng: So what does that mean in the context of deployment? Does this complexity mean we’re looking at a massive increase in computational cost for running the model on real traffic?

Tom: The paper shows that it consistently outperforms representative single-modality and multimodal segmentation baselines on both PICCOLO and FAHSYSU datasets, achieving the best region-level segmentation performance.

Jane: So the implication is that we get much better boundary delineation, which is what matters clinically. It’s about not just getting a rough idea of where the lesion is, but getting a precise contour.

Lu: The configuration using WLI-derived features to parameterize magnitude and NBI-derived features to parameterize the phase component consistently achieves the best performance on both datasets.

Meng: So it means that this specific way of separating appearance and structure is actually what drives the best results in this setting.

Tom: Yeah, so we’ve seen how coupling diffeomorphic feature registration with complex spectral fusion helps here. That’s a solid summary of "Registration-Grounded Spectral Fusion for Unregistered WLI/NBI Endoscopic Lesion Segmentation".

School of Intelligent Systems Engineering, Shenzhen Campus of Sun Yat-sen University · Department of Otolaryngology, The First Affiliated Hospital of Sun Yat-sen University · School of Computer Science and Technology, Hainan University · School of Mathematics and Statistics, Xi’an Jiaotong University · Chongqing University of Posts and Telecommunications

cs.CV

Submitted: 2026-07-29

Updated: 2026-10-20

Comments: 15 pages

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 79/100

The gist: The gist The proposed framework addresses paired-but-unregistered WLI/NBI lesion segmentation by coupling topology-regularized correspondence grounding with reliability-guided complex-domain

Key concepts

Reliability-Aware Diffeomorphic Registration Grounding (R-DRG): This module finds the spatial relationship between WLI and NBI features using smooth, invertible transformations that preserve topology. Crucially, it estimates a reliability map showing where this cross-modal correspondence is trustworthy. This prevents merging incorrect areas during fusion.
Role-Aware Complex Spectral Fusion (R-CSF): This fusion method treats WLI and NBI features differently in the complex plane. It models WLI cues as magnitude responses (appearance) and NBI cues as phase responses (structure). Reliability maps control how strongly these roles interact, suppressing errors in mismatched regions.
Topology-Preserving Feature Mapping: The registration process uses bidirectional diffeomorphic transformations derived from a stationary velocity field. This ensures the geometric mapping between the two modalities is smooth and maintains the underlying structure of the tissue, effectively compensating for viewpoint changes and deformation.

Terminology

Ablation — 1× in this paper · explained in 1 episode
Ablation Studies — 1× in this paper · explained in 1 episode
Grounding — 1× in this paper · explained in 3 episodes
Multimodal Fusion — 1× in this paper · explained in 4 episodes

Summary

The gist The proposed framework addresses paired-but-unregistered WLI/NBI lesion segmentation by coupling topology-regularized correspondence grounding with reliability-guided complex-domain interaction to exploit modality complementarity while suppressing unreliable cross-modal interaction.

Problem and Motivation

White-light imaging (WLI) and narrow-band imaging (NBI) provide complementary views of endoscopic lesions, but their paired observations are often spatially misaligned due to viewpoint changes, tissue deformation, and sequential handheld acquisition <ref:2607.26395#pg2> The lack of task-relevant pixel-topixel correspondence makes direct WLI/NBI fusion prone to mixing non-corresponding regions and may even degrade segmentation around lesion boundaries <ref:2607.26395#pg2> This difficulty highlights a gap between clinical dual-modality observation and current automatic segmentation practice <ref:2607.26395#pg2> Existing methods mainly improve feature compatibility or introduce coarse alignment, but they do not explicitly establish topology-regularized feature correspondence together with local correspondence reliability before cross-modal interaction <ref:2607.26395#pg2> Furthermore, most WLI/NBI fusion methods aggregate features in the spatial or real-valued domain and often treat WLI and NBI as exchangeable information sources <ref:2607.26395#pg2> Therefore, effective WLI/NBI segmentation requires reliability-aware correspondence modeling and role-aware complementary fusion <ref:2607.26395#pg2>.

Proposed Framework Components

The framework consists of a modality-aware pyramid encoding, Reliability-Aware Diffeomorphic Registration Grounding (R-DRG), Reliability-Guided Complex Spectral Fusion (R-CSF), and dual-branch segmentation decoding <ref:2607.26395#pg2>.

Reliability-Aware Diffeomorphic Registration Grounding (R-DRG) establishes topologyregularized cross-modal correspondence and estimates where crossmodal interaction is reliable before spectral fusion <ref:2607.26395#pg2>
R-DRG performs registration in feature space, estimating a stationary velocity field and bidirectional diffeomorphic transformations to obtain smooth, invertible, and topologypreserving cross-modal correspondence <ref:2607.26395#pg2>
It computes correspondence reliability maps from the projected feature discrepancy after warping to guide subsequent fusion <ref:2607.26395#pg2>

Reliability-Guided Complex Spectral Fusion (R-CSF)

R-CSF performs reliability-guided magnitude–phase interaction in a learnable complex feature space, where WLI-dominant appearance cues and NBI-dominant structural cues are selectively fused according to local correspondence reliability <ref:2607.26395#pg2>

It constructs modality-role-aware polar representations, where WLI-derived cues mainly parameterize magnitude-like appearance responses and NBI-derived cues mainly parameterize phase-like structural responses <ref:2607.26395#pg2>
The reliability maps are broadcast along the channel dimension to decide how strongly cross-modal spectral cues should be injected, allowing the model to fall back to modality-specific cues when cross-modal correspondence is unreliable <ref:2607.26395#pg2>
The branch-specific complex features are then constructed as Zlwi and Zlnbi, which are transformed into the frequency domain for frequency-selective complex recalibration using radial frequency masks <ref:2607.26395#pg2>

Joint Optimization and Results

The overall training objective is formulated as a combination of segmentation loss, smoothness regularization on the velocity field, probability-level and lesion-support consistency constraints, and the reliability maps are used only as fusion priors in R-CSF <ref:2607.26395#pg2>

The framework is jointly optimized under the final segmentation objective using a weighted sum of branch-wise segmentation supervision for both modalities <ref:2607.26395#pg2>
Regularization terms include smoothness regularization on the stationary velocity field to encourage smooth deformation and probability-level constraints comparing probability masks after geometric transformation <ref:2607.26395#pg2>
The complete model consistently outperforms representative single-modality and multimodal segmentation baselines on both PICCOLO and FAHSYSU datasets, achieving the best region-level segmentation performance <ref:2607.26395#pg2>.

Key Findings

The proposed method achieves superior performance by explicitly modeling the different roles of WLI and NBI, as evidenced by Table 4 showing that the configuration using WLI-derived features to parameterize the magnitude component and NBI-derived features to parameterize the phase component consistently achieves the best performance on both datasets <ref:2607.26395#pg2>. Furthermore, boundary-sensitive evaluation metrics like HD95 and ASSD show that our method achieves the best ASSD and BIoU on PICCOLO, and ranks second in HD95 only behind ADFNet <ref:2607.26395#pg2>. The visualization of the learned deformation shows that after warping, the NBI feature becomes more consistent with the WLI feature in both lesion contour and activation distribution <ref:2607.26395#pg2>.

Conclusion

The proposed framework provides a principled fusion paradigm for paired-but-unregistered WLI/NBI segmentation by coupling topology-regularized correspondence grounding with role-aware complex spectral fusion, which effectively reduces negative transfer from locally mismatched WLI/NBI observations <ref:2607.26395#pg2>. The results support the effectiveness of coupling diffeomorphic feature registration with complex spectral fusion in this challenging setting <ref:2607.26395#pg2>.

AI Assistance Note

During the preparation of this work, the authors used ChatGPT (OpenAI) to assist with language polishing, grammatical correction, consistency checking, and improving the clarity and readability of the manuscript <ref:2607.26395#pg2>. All methodological design, scientific hypotheses, experimental results, data analyses, statistical testing, interpretation, conclusions, citations, and references were manually designed, performed, checked, and verified by the listed authors without the involvement of any large language model <ref:2607.26395#pg2>.

References

[1] Cao et al., 2021. Shapeconv: Shape-aware convolutional layer for indoor rgb-d semantic segmentation, in: Proceedings of the IEEE/CVF international conference on computer vision, pp. 7088–7097 <ref:2607.26395#pg2>

[2] Cao et al., 2025. Multimodal image generation and fusion through content-style hybrid disentanglement, Knowledge-Based Systems 330, 114597 <ref:2607.26395#pg2>

[3] Chen et al., 2024a. Weakly misalignment-free adaptive feature alignment for uavs-based multimodal object detection, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 26836–26845 <ref:2607.26395#pg2>

[4] Chen et al., 2025a. A survey on deep learning in medical image registration: New technologies, uncertainty, evaluation metrics, and beyond, Medical Image Analysis 100, 103385 <ref:2607.26395#pg2>

[5] Chen et al., 2025b. Unsupervised learning of spatially varying regularization for diffeomorphic image registration, Medical image analysis, 103887 <ref:2607.26395#pg2>

[6] Chen et al., 2024b. Frequency-aware feature fusion for dense image prediction, IEEE Transactions on Pattern Analysis and Machine Intelligence 46, 10763–10780 <ref:2607.26395#pg2>

[7] Chen et al., 2026. Joint multiview embedding with progressive multi-scale alignment for unaligned infrared-visible image fusion, Information Fusion 128, 103960 <ref:2607.26395#pg2>

[8] Chen et al., 2025c. Evaluation of band selection for spectrum-aided visual enhancer (save) for esophageal cancer detection, Journal of Cancer 16, 470 <ref:2607.26395#pg2>

[9] Dong et al., 2025. A fusion network for multimodality medical image registration with progressive feature alignment, Knowledge-Based Systems 317, 113427 <ref:2607.26395#pg2>

[10] Fan et al., 2024. Bilevel learning of task-specific decoders for joint registration and oneshot medical image segmentation, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 11726–11735 <ref:

Improvements for AI systems

Reliability-Aware Feature Correspondence Estimation (R-DRG): Develop a module that establish[es] topology-regularized feature correspondence and estimate[s] where the crossmodal correspondence is reliable, which prevents residual local mismatch may still cause unreliable feature fusion around lesion boundaries and fine mucosal structures.
Role-Aware Complex Spectral Fusion (R-CSF): Implement a mechanism that explicitly model[s] the different roles of WLI and NBI by allowing WLI-derived cues mainly provide appearance-related magnitude responses, while NBI-derived cues provide structure-sensitive phase responses, suppressing unreliable interaction in locally mismatched regions.
Reliability-Guided Spectral Recalibration: Integrate reliability maps to control spectral intensity and angular response via the equation provided in (8), ensuring that High reliability regions are allowed to receive stronger spectral recalibration, whereas low-reliability regions preserve more of the original modality-specific complex feature.
Topology-Preserving Feature Mapping: Utilize bidirectional diffeomorphic transformations derived from a single SVF parameterization (equation 3) to provide a smooth and topologypreserving geometric basis before crossmodal fusion, thereby mitigating issues from viewpoint changes, tissue deformation, and sequential handheld acquisition.
Boundary-Sensitive Delineation: Leverage the framework's output to achieve superior boundary quality, as evidenced by achieving the best ASSD and BIoU on PICCOLO data, indicating that the method improves not only region overlap but also contour-level lesion delineation.