DiffEEG: A Self-Supervised Denoising Diffusion Model for Learning EEG seizure-related Representations

arXiv:2607.11578 · cs.LG, cs.AI, cs.CV, eess.SP · Submitted 2026-07-13 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Hi, I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: We're looking at the paper: "DiffEEG: A Self-Supervised Denoising Diffusion Model for Learning EEG seizure-related Representations".

Jane: Here's the gist of the paper. The gist The DiffEEG model achieves clinically viable seizure monitoring by integrating denoising diffusion pre-training and reinforcement learning to address annotation scarcity and extreme class imbalance in EEG decoding.

Tom: How it works DiffEEG is organized as a two-stage framework where the first stage involves self-supervised diffusion pre-training, and the second stage adapts the model for seizure classification through a reinforced decision layer.

Jane: The pre-trained encoder is built using a 1D U-Net architecture augmented with multi-head self-attention and FiLM conditioning to learn generic neural representations from unlabeled EEG data.

Tom: This process utilizes a forward noising process and a learned reverse denoising process defined by the equations for the forward and reverse processes.

Paper summary: Tom: So, wrapping up this discussion on DiffEEG, the authors are presenting this self-supervised denoising diffusion model for learning EEG seizure-related representations >

Jane: It seems like the main takeaway is that by combining diffusion pre-training and reinforcement learning to address annotation scarcity and extreme class imbalance in EEG decoding, they've managed to create something that could be clinically deployed with minimal labeled data requirements >

Lu: The model essentially proves that you can build a powerful foundation model for EEG representations on unlabeled data, and then use RL to specifically guide its decision-making towards the rare events >

Meng: From a deployment standpoint, it means we might not need millions of manually labeled seizure recordings to get something that works at scale, though I'd still want to see how this nine point six million parameter model handles real-world noise and patient variability in a production environment >

Lalam: The authors are showing remarkable results on imbalanced data for seizure subtyping and a binary detection on the TUHSZ corpus >

Tom: So, the implication is that this approach opens up the possibility of having more reliable, automated seizure monitoring tools based on less clinical annotation effort >

Conclusion: Tom: So, we're finishing up on DiffEEG, this self-supervised model for seizure detection. The title is "DiffEEG: A Self-Supervised Denoising Diffusion Model for Learning EEG Seizure-Related Representations."

Jane: That title sounds a bit heavy with all those technical terms. What’s the actual core idea here? How do we simplify what this paper is trying to do?

Tom: The core idea is using denoising diffusion pre-training and reinforcement learning to deal with the massive problem of not having enough labeled EEG data for seizures.

Lu: Exactly. It’s about taking a lot of unlabeled EEG data, like the one point three million segments they used from Temple University Hospital, and teaching the model to understand what a seizure looks like without needing millions of hand-labeled examples upfront.

Meng: Okay, so it's learning generic neural representations first on the big unlabeled set? What’s the actual mechanism behind that denoising diffusion pre-training? How does it learn those features?

Tom: The encoder is built using a one-dimensional U-Net architecture, but they add multi-head self-attention and FiLM conditioning. That FiLM part modifies the feature learning based on a timestep embedding which tells the model how noisy or clear the input data is.

Jane: So it’s like teaching the model to clean up noise and understand EEG patterns by slowly adding and then removing noise during this diffusion process?

Lu: Right. They use a forward noising process, then they learn a reverse denoising process defined by specific equations for that forward and reverse steps. It forces the model to learn really robust features because it has to predict the original signal from the noisy one.

Jane: That makes sense for learning representations on unlabeled data. But how do we actually get it to make a seizure detection decision? That’s where the reinforcement learning comes in, right?

Tom: Yeah. The second stage is adaptation using a reinforced decision layer, or RDL. This part uses policy gradient optimization to directly try and maximize the F1-score on the seizure events themselves.

Meng: Policy gradient optimization sounds like a way to tweak the model's output adjustments based on rewards, right? What’s the actual loss function they’re minimizing then?

Lu: They combine a standard weighted cross-entropy loss with a policy gradient auxiliary term. The policy gradient part uses the REINFORCE algorithm with a moving-average baseline to keep the gradients from getting too noisy. That's how it guides the model toward prioritizing sensitivity for those rare events.

Jane: So, they’re not just classifying; they’re actively training the decision layer to be extra sensitive where seizures are happening, even when data is scarce?

Tom: Precisely. And on testing, they got sixty-one percent accuracy and a fifty-nine percent F1 for four-class subtyping using only patient-wise evaluation, which is pretty strict Leave-One-Fold-Out testing.

Lu: That patient variability was tricky, though. They noted that performance varied across folds because electrode placements and individual seizure signatures change so much between patients.

Meng: So the RL loss was actually important for stabilizing things in those high-variance scenarios, even when the main cross-entropy loss started to level off? What’s a practical takeaway for deployment then?

Tom: The practical thing is they showed that this diffusion pre-training combined with metric-aware reinforcement learning lets you get clinically relevant results with minimal labeled data requirements. It moves the goalpost on how much labeled data we actually need for these kinds of medical AI tasks.

Jane: It’s a big deal because it suggests we can build tools for rare events, like seizures, without needing massive, expensive clinical datasets to start. So where do we go from here?

Lu: The authors are looking ahead at testing this model across more diverse datasets to really prove that its performance holds up when you move it outside the specific hospital corpus they trained it on.

Abdulkader Helwan, Lina Abou-Abbas, *Hussein El Amouri, Belkacem Chikhaoui, Khadidja Henni

Department of Electrical and Computer Engineering, Lebanese American University · Institute of Applied Artificial Intelligence, TELUQ University

cs.LG, cs.AI, cs.CV, eess.SP

Submitted: 2026-07-13

Updated: 2026-10-20

Comments: 19 pages, 6 figures

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

The gist: The gist The DiffEEG model achieves clinically viable seizure monitoring by integrating denoising diffusion pre-training and reinforcement learning to address annotation scarcity and extreme class

Key concepts

Denoising Diffusion Pre-training: This is a self-supervised training method where the model learns to reverse a gradual noise process applied to EEG signals. It helps the model learn general, high-quality features from massive amounts of unlabeled data without needing manual labels for every segment.
Reinforcement Learning (RL)-based Fine-tuning: After pre-training, a reinforcement learning layer is added. This layer uses policy gradient optimization to adjust the model's output to directly maximize the F1-score, specifically prioritizing sensitivity for rare seizure events over overall accuracy.
Policy Gradient Optimization: This is an RL technique used during fine-tuning. It treats the classification decision as a policy that can be adjusted. The optimization aims to find adjustments that increase the desired outcome (like F1-score) by iteratively updating the model's parameters based on rewards.
Class Imbalance Handling: The model is specifically designed to handle extreme imbalance, where seizure events are very rare (less than 10% of recordings). By using RL to maximize the F1-score, DiffEEG ensures it maintains good recall for these critical, rare events.

Terminology

Accuracy — 4× in this paper · explained in 1 episode
Reinforcement Learning — 2× in this paper · explained in 4 episodes
Adaptation — 1× in this paper · explained in 1 episode
Class Imbalance — 1× in this paper · explained in 2 episodes
Diffusion Model — 1× in this paper · explained in 10 episodes
Fine-Tuning — 1× in this paper · explained in 5 episodes
Foundation Model — 1× in this paper · explained in 3 episodes
Reinforcement Learning (RL) — 1× in this paper · explained in 25 episodes

Summary

The gist The DiffEEG model achieves clinically viable seizure monitoring by integrating denoising diffusion pre-training and reinforcement learning to address annotation scarcity and extreme class imbalance in EEG decoding.

How it works

DiffEEG is organized as a two-stage framework where the first stage involves self-supervised diffusion pre-training, and the second stage adapts the model for seizure classification through a reinforced decision layer. The pre-trained encoder is built using a 1D U-Net architecture augmented with multi-head self-attention and FiLM conditioning to learn generic neural representations from unlabeled EEG data. This process utilizes a forward noising process and a learned reverse denoising process defined by the equations for the forward and reverse processes.

The second stage focuses on adaptation using a reinforced decision layer (RDL) that employs policy gradient optimization to directly maximize the F1-score. The RDL parameterizes a policy network πϕ: R D → R that maps the aggregated feature vector to a scalar logit adjustment, which is applied asymmetrically to the classifier logits 0 increase sensitivity toward seizure detection>.

Model Architecture and Training Objectives

The core neural network employs a U-Net-based encoder-decoder architecture augmented with multi-head self-attention mechanisms and feature-wise linear modulation (FiLM) conditioning. The residual blocks incorporate FiLM conditioning, where the timestep embedding temb is processed through a multilayer perceptron before FiLM conditioning applies h'' = h' · (1 + γ) + β.

The total loss function combines weighted cross-entropy with a policy gradient auxiliary term, defined as L = LCE + λLPG, where λ = 0.1. The policy gradient loss is calculated using the REINFORCE algorithm with a moving-average baseline to reduce gradient variance.

Performance and Clinical Validation

Under strict patient-wise evaluation (279 patients, Leave-One-Fold-Out), DiffEEG achieves 61% accuracy and 59% F1 for 4-class seizure subtyping, and 81% accuracy with 85% weighted F1 for binary detection. Segment-level evaluation establishes an upper bound of 97.6% accuracy, confirming strong architectural capacity. The reinforcement learning auxiliary loss was activated after epoch 5 (warmup period), and as shown in Figure 5, the RL reward (macro F1) consistently improved after activation, even when cross-entropy loss plateaued.

Addressing Data Scarcity and Generalization

The framework addresses label scarcity through several strategies. For seizure subtype classification, a key technical contribution is the reuse of the discriminative head from the binary detection task, which preserved high-level discriminative features. Furthermore, few-shot learning (K-Shot Analysis) reveals that while frozen backbone strategies provide a reliable baseline, the unfrozen (partial fine-tuning) strategy consistently outperformed the frozen backbone at the highest shot configuration.

Patient-wise cross-validation results showed performance variability across folds, peaking at 67.38% F1 in Fold 2, reflecting the challenges posed by differing electrode placements and individual seizure signatures. The integration of the policy gradient (RL) auxiliary loss was critical to stabilize performance in these high variance scenarios, ensuring the model remains clinically relevant for identifying rare but medically significant events.

Benchmarking and Comparison

DiffEEG demonstrates superior performance compared to state-of-the-art supervised baselines in segment-wise classification, achieving an Accuracy of 97.66% and a Weighted F1-score of 97.67%. In binary seizure detection, DiffEEG achieved a Weighted F1-score of 0.85 and a macro F1-score of 59%, maintaining clinically viable seizure recall (59%) despite extreme imbalance. The model also showed superior performance on the TUAB dataset for clinical abnormality detection, achieving a state-of-the-art AUROC of 0.922.

Conclusion The paper proposes DiffEEG, a self-supervised foundation model that integrates denoising diffusion pre-training with a RDL to address label scarcity and class imbalance in EEG decoding >. This approach enables clinically deployable seizure monitoring with minimal labeled data requirements >. The model achieves remarkable results on imbalanced data for seizure subtyping and a binary detection on the TUHSZ corpus >. Future research will focus on testing the model across more diverse datasets to further validate its cross-dataset robustness and clinical reliability >.

--- Page 1 ---