Neural Visual Decoding via Cognitive guided Adaptive Blurring and Information Constrained Alignment

The gist: CAIA proposes a Cognitive-guided Adaptive blurring with Information-Constrained Alignment framework for Neural-Visual decoding, which improves both subject-dependent and subject-independent average Top-1 and Top-5 accuracy in zero-shot brain-toimage retrieval by dynamically fusing visual cues via EEG cross-modal attention and enhancing SNR through information screening.

How it works

CAIA bridges modality gaps via bidirectional information modulation, simulating selective attention on the visual side and leveraging neural oscillation priors on the EEG side. Visually, a cognitive-dynamics-based blurring mechanism combines center-fixation and saliency-guided paths, which are dynamically fused via an EEG-driven cross-modal attention module to simulate selective attention and reduce information density

The dual blurring mechanism comprises two parallel paths:

1. Saliency-Guided Differential Blurring, which applies spatially varying blur intensity inversely proportional to a saliency map, generating the blurred image XA.

2. Center-Guided Radial Blurring, which uses a radial blur profile that preserves clarity at the center and gradually increases blur toward the periphery based on normalized distance from the image center.

These two paths are fused using a cross-modal attention strategy where EEG features serve as the query Q, and concatenated dual-path blurred images form the key K to estimate pixel-wise fusion weights A, resulting in Xfused = A ⊙ XA + (1 − A) ⊙ XB.

Information Enhancement Modules

The EEG side of CAIA incorporates two mechanisms to enhance signal quality and relevance:

1. Cognitively guided information-screening method, which integrates neural oscillation priors with the Information Bottleneck (IB) principle to select task-relevant frequency bands. This involves learning a compressed representation Z that preserves information about the visual stimulus Y while discarding task-irrelevant variability from the raw EEG signal X. The objective is formulated as min p(zx) I(X;Z) − βI(Z; Y).

2. Distribution-aware boundary calibration loss, which dynamically identifies outliers based on similarity statistics and softly draws them toward a plausible similarity range to improve robustness to attention shifts and noise. This loss is defined by Lbound = 1/M Σ X i∈Ml left i + l right i.

Experimental Validation and Performance

Extensive experiments demonstrate that CAIA improves both subject-dependent and subject-independent average Top-1 and Top-5 accuracy in zero-shot brain-toimage retrieval, significantly outperforming prior methods. On THINGS-EEG, CAIA attains a subject-dependent Top-1/Top-5 accuracy of 70.3% / 96.1%, surpassing the strong UBP baseline by +19.4% / +16.4%. In the more challenging subject-independent setting, CAIA achieves 16.4% / 45.7% (UBP: 12.4% / 33.4%), demonstrating enhanced robustness to inter-subject variability through the distribution-aware boundary loss. Ablation studies confirm that the full CAIA model achieves the best overall performance of 70.3% Top-1 and 96.1% Top-5 accuracy.

Analysis of Components

Visual analyses confirm effective modeling of attentional selectivity, as the attention heatmap inferred by the adaptive blurring mechanism aligns with image semantics, avoiding uniform blurring or overreliance on a single pathway. The frequency filter comparison shows that IBWave outperforms static band-pass filters because it dynamically optimizes both decomposition scales and band-selection weights in a task-driven manner, confirming the gains stem from task-adaptive band selection rather than generic denoising. Furthermore, the distribution-aware boundary loss effectively calibrates the EEG-image similarity distribution by increasing its compactness and mean similarity while reducing outliers beyond the confidence region. The computational overhead analysis shows that CAIA’s inference latency of 14.29 ms remains well within practical constraints for real-time BCI systems, constituting only a modest increase over UBP.

Conclusion and Limitations

CAIA presents a cognitively guided framework that mitigates the information granularity mismatch in EEG-visual decoding via bidirectional modulation, effectively fusing center-biased and saliency-guided cues while enhancing SNR through information screening and outlier calibration. A limitation noted is that the adopted cognitive priors—center bias and saliency capture—presuppose an experimental paradigm with sustained central fixation, meaning their effectiveness may degrade under free-viewing conditions or paradigms without a well-defined fixation point. This work offers a principled, cognition-informed approach to EEG-visual alignment, advancing both performance and interpretability. The framework successfully establishes robust cross-modal alignment by jointly modulating visual and neural representations while explicitly modeling attentional dynamics.

--- Page 1 ---

The gist: CAIA proposes a Cognitive-guided Adaptive blurring with Information-Constrained Alignment framework for Neural-Visual decoding, which improves both subject-dependent and subject-independent average Top-1 and Top-5 accuracy in zero-shot brain-toimage retrieval by dynamically fusing visual cues via EEG cross-modal attention and enhancing SNR through information screening.

How it works

CAIA bridges modality gaps via bidirectional information modulation, simulating selective attention on the visual side and leveraging neural oscillation priors on the EEG side. Visually, a cognitive-dynamics-based blurring mechanism combines center-fixation and saliency-guided paths, which are dynamically fused via an EEG-driven cross-modal attention module to simulate selective attention and reduce information density.

The dual blurring mechanism comprises two parallel paths:

1. Saliency-Guided Differential Blurring, which applies spatially varying blur intensity inversely proportional to a saliency map, generating the blurred image XA.

2. Center-Guided Radial Blurring, which uses a radial blur profile that preserves clarity at the center and gradually increases blur toward the periphery based on normalized distance from the image center.

These two paths are fused using a cross-modal attention strategy where EEG features serve as the query Q, and concatenated dual-path blurred images form the key K to estimate pixel-wise fusion weights A, resulting in Xfused = A ⊙ XA + (1 − A) ⊙ XB.

Experimental Validation and Performance

Extensive experiments demonstrate that CAIA improves both subject-dependent and subject-independent average Top-1 and Top-5 accuracy in zero-shot brain-toimage retrieval, significantly outperforming prior methods. On THINGS-EEG, CAIA attains a subject-dependent Top-1/Top-5 accuracy of 70.3% / 96.1%, surpassing the strong UBP baseline by +19.4% / +16.4%.

1. The CAIA framework can be implemented to bridge modality gaps via bidirectional information modulation, where EEG features serve as the query Q, encoding the subject’s cognitive state, while features from the concatenated dual-path blurred images form the key K. This allows for a dynamic fusion of visual cues informed by neural signals.

2. The system can employ a cognitively guided information-screening method to enhance Signal-to-Noise Ratio (SNR) by selecting task-relevant frequency bands, as evidenced by the finding that the Beta band receives the highest selection weight. This allows for systematic suppression of irrelevant oscillations, leading to better decoding in challenging conditions.

3. The distribution-aware boundary calibration loss can be integrated to improve robustness against noise and attention shifts by identifying outliers where similarity lies near the distribution mean as 'inliers,' while those deviating significantly—often due to attention drift or high-noise trials—are treated as 'outliers.' This dynamically guides samples toward a plausible similarity range.

4. The cognitive-dynamics-based adaptive blurring mechanism can simulate selective attention by using a dual-path adaptive blurring mechanism that dynamically selects and retains perceptually relevant image regions, moving beyond static blurring strategies. This allows the system to adapt its visual representation based on inferred attentional states.

EEG-based visual decoding aims to establish a mapping between neural signals and visual semantics. However, it remains constrained by the dual challenges of severe information granularity mismatch and the low signal-to-noise ratio (SNR) of EEG signals. Existing approaches typically treat visual features as static, ignoring the dynamic selectivity of human vision and the frequency specificity of neural oscillations. To bridge this gap, we propose CAIA, a Cognitive-guided Adaptive blurring with Information-Constrained Alignment framework for Neural-Visual decoding. On the visual side, it simulates selective attention to adaptively reduce redundancy. Meanwhile, on the EEG side, it combines neural oscillation priors with an information-bottleneck-inspired spectral sparsity regularizer to enhance SNR. Specifically, we devise a cognitive-dynamics-based adaptive blurring mechanism that dynamically integrates center-biased and saliency-guided visual cues via cross-modal attention. Furthermore, we introduce a distribution-aware boundary calibration loss to robustly rectify alignment bias caused by outlier samples. Moreover, a cognitively guided information-screening method is proposed to select task-relevant EEG oscillations. Extensive experiments demonstrate that CAIA improves both subject-dependent and subject-independent average Top-1 and Top-5 accuracy in zero-shot brain-to-image retrieval, significantly outperforming prior methods. Our work validates that optimizing visual information density to match neural granularity offers a more interpretable and robust pathway for neural decoding.