Deep learning-based multi-level airway collapse site localization on obstructive sleep apnea patients based on snoring sounds

arXiv:2408.16030 · cs.SD, cs.AI, cs.LG, eess.AS · Submitted 2024-08-28 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "Deep learning-based multi-level airway collapse site localization on obstructive sleep apnea patients based on snoring sounds".

Jane: Here's the gist of the paper. The gist The study investigated deep learning models to identify multi-level upper airway collapses in obstructive sleep apnea patients using snoring sounds, finding that Audio Spectrogram Transformer (AST) slightly outperformed ResNet-50 in identifying V and O level obstructions, while both models struggled with T, E, and RG classifications due to limited data.

Tom: Data Collection and Labeling The researchers used snoring recordings from 37 subjects undergoing drug-induced sleep endoscopy (DISE) between 2020 and 2021 to train their models<ref:2408.16030#pg6>.

Jane: Snoring sounds were labeled according to the VOTE (Velum, Orophar-ynx, Tongue Base, Epiglottis) classification system, resulting in specific counts for each class including 403 O and 77 T labels<ref:2408.16030#pg6>.

Tom: The data preparation involved synchronizing endoscopic video tapes with breath sounds and aligning the snoring labels (Set A) and VOTE labels (Set B)<ref:2408.16030#pg9>.

Paper summary: Tom: So we’re looking at this paper today called "Deep learning-based multi-level airway collapse site localization on obstructive sleep apnea patients based on snoring sounds." It tackles identifying where those collapses are happening in people with sleep apnea using just snoring sounds.

Jane: That’s the main goal, Tom. The study uses deep learning models, specifically ResNet-fifty and an Audio Spectrogram Transformer called AST, to pinpoint multi-level upper airway collapses from snoring recordings of patients with obstructive sleep apnea <ref:2408.16030#pg3,multi-level upper airway collapses>.

Lu: What’s interesting about this approach is that they are training these models on data gathered from drug-induced sleep endoscopy, or DISE, which gives them a pretty direct look at the airways.

Meng: So what do they claim? Basically, they fine-tuned ResNet-fifty and AST to classify obstructions at four VOTE levels: Velum, Oropharynx, Tongue Base, and Epiglottis <ref:2408.16030#pg3>. They also looked for retropalatal and retroglossal regions.

Jane: And the results show that the Audio Spectrogram Transformer model actually outperformed ResNet-fifty in identifying obstructions at the Velum level with an F1-score of zero point seven one compared to zero point six four, and it also did better for Oropharynx obstructions with an F1-score of zero point eight zero versus a zero point seven two for ResNet-fifty <ref:2408.16030#pg3>.

Tom: That’s a solid performance gap on the V and O levels, which is encouraging because those are often considered key areas for OSA assessment in snoring analysis. But then they hit a wall with the Tongue Base and Epiglottis classifications <ref:2408.16030#pg3>.

Lu: They did struggle there, which makes sense considering the data limitations; they mentioned that both models struggled with T, E, and RG classifications because of small sample sizes even after using synthetic minority over-sampling technique.

Jane: So, the study is saying that while AST is better than ResNet-fifty for V and O levels—reaching an AUC of zero point eight nine for Velum obstructions <ref:2408.16030#pg3>—the other levels like Tongue Base, Epiglottis, and the RP or RG regions remain difficult to classify reliably right now.

Meng: From a deployment standpoint, that limitation is huge. If we can’t accurately pinpoint those lower levels with the current data size, it means we can't build a comprehensive diagnostic tool yet for all parts of the airway using just these sounds <ref:2408.16030#pg3>.

Tom: Exactly. But the paper doesn't stop there; they also did a retrospective analysis of a full-night recording to see if they could track dynamics over time and find some potential for that.

Lu: That part is where the ambition really comes through, because they looked at using a previously proposed convolutional bidirectional gated recurrent network model to profile inhalation, exhalation, and continuous adventitious sound episodes from those full-night recordings <ref:2408.16030#pg16>.

Jane: And what did that full-night analysis suggest? They found that the model predicted the snoring sounds mainly originated at the Velum level, followed by Oropharynx, with a slight rise in Epiglottis association in the last half hour of sleep <ref:2408.16030#pg18>.

Tom: That’s a shift from just classifying individual sound clips to understanding the pattern over time during sleep. It suggests that even if we miss some specific sites like T or E, we can still get a general idea of where the trouble is occurring across the night.

Meng: If we were actually putting this into a product, I’d be worried about how much data you need to make those full-night predictions robust enough to deploy reliably on real patient traffic. The paper suggests that longitudinal tracking *could* work, but scaling that prediction reliably is the next big hurdle <ref:2408.16030#pg18>.

Lu: I think the core idea is moving from just static classification to dynamic profiling, which is a much richer way to look at airway mechanics. The model's ability to profile the occupation rate of snoring by feeding segments into that VOTE model shows promise for tracking dynamics <ref:2408.16030#pg16>.

Jane: So, if you’re listening and you’re thinking about what this means for sleep apnea patients, it points toward a tool that can monitor things continuously rather than just taking snapshots of a sound clip.

Tom: And the title itself, "Deep learning-based multi-level airway collapse site localization on obstructive sleep apnea patients based on snoring sounds," tells you exactly what this research is trying to achieve with these models and these specific patient data.

Lu: It's about using deep learning to localize those collapses across multiple levels—VOTE and RP/RG—using the acoustic information from snoring sounds.

Jane: So, to wrap up, the study shows that while they have a strong model called AST beating ResNet-fifty for identifying obstructions at the Velum and Oropharynx levels, they still have significant challenges classifying Tongue Base, Epiglottis, and RP or RG regions due to limited data <ref:2408.16030#pg3>.

Meng: The implication for me is that if we want to build something deployable today, we need more variety in the training data for those harder-to-classify sites before we can claim this technology works universally <ref:2408.16030#pg3>.

Tom: It’s a promising non-invasive tool for tracking overnight airway dynamics and assisting in clinical treatment planning, as the paper concludes.

Lu: That sounds like it could help clinicians adjust time slices to fit regions of interest and synchronize those findings with things like PSG data <ref:2408.16030#pg18>.

Jane: So, this paper gives us a concrete direction for developing AI tools that can move beyond simple detection to actually mapping out the complexity of an airway collapse over time.

Conclusion: Tom: So we’ve looked at how AST outperformed ResNet-fifty on those V and O level identifications, and now we’re getting to the conclusion of this paper, "Deep learning-based multi-level airway collapse site localization on obstructive sleep apnea patients based on snoring sounds."

Jane: That title is pretty descriptive, isn't it? It tells you exactly what they did: use deep learning to find where the airway collapses are happening in people with sleep apnea by analyzing snoring sounds.

Lu: The authors used thirty-seven subjects from drug-induced sleep endoscopy data to train their models for these multi-level tasks across the VOTE levels and the RP or RG regions.

Meng: What’s important here is that they didn't just look at one thing, they tried to map out four different areas of the airway obstruction simultaneously.

Tom: And what they found is that while AST was better than ResNet-fifty for those top two levels—Velum and Oropharynx—it still hit a wall with the Tongue Base, Epiglottis, and Retroglossal regions because of how little data they had for those specific spots.

Jane: So, what does that mean in plain terms? It means the AI is pretty good at spotting trouble high up in the airway structure based on snoring sounds right now.

Lu: Exactly. The authors are showing that this acoustic profiling method has potential because it can track dynamics over a full night recording, which is a big step for understanding how sleep apnea progresses.

Tom: Tracking dynamics means you move beyond just saying "there's an obstruction" to figuring out *when* and *where* the obstruction shifts during sleep.

Jane: That’s the part where it gets interesting for clinical use. It suggests that these models could become a non-invasive way for doctors to track airway health overnight and help plan treatments before a surgery or procedure.

Meng: From my side, the challenge is getting that system to run reliably on real patient data without breaking at scale, especially since the authors mentioned those lower levels are still weak spots.

Lu: That's the next big question for future work. Can we expand that training set with more diverse snoring sounds so these models can get better at identifying those harder-to-spot areas?

Ying-Chieh Hsu, Stanley Yung-Chuan Liu, Chao-Jung Huang, Chi-Wei Wu, Ren-Kai Cheng, Jane Yung-Jen Hsu, Shang-Ran Huang, Yuan-Ren Cheng, Fu⶟⶟ Hsu

Department of Biomedical Engineering, National Taiwan University · Department of Otolaryngology Head and Neck Surgery, Taipei Tzu Chi Hospital, Buddhist Tzu Chi Medical Foundation · Tzu Chi University, School of Medicine · Department of Oral & Maxillofacial Surgery, Nova Southeastern University College of Dental Medicine and College of Allopathic Medicine · All Vista Healthcare Center and Center for AI and Advanced Robotics, National Taiwan University · Department of Computer Science and Information Engineering, National Taiwan University · Department of Artificial Intelligence, Chang Gung University College of Intelligent Computing · Heroic Faith Medical Science Co., Ltd. · School of Medicine, Chung Shan Medical University · Hospital Development Affairs Office, Chung Shan Medical University Hospital

cs.SD, cs.AI, cs.LG, eess.AS

Submitted: 2024-08-28

Updated: 2026-10-20

License: http://creativecommons.org/licenses/by-sa/4.0/

Importance score: 68/100

The gist: The gist The study investigated deep learning models to identify multi-level upper airway collapses in obstructive sleep apnea patients using snoring sounds, finding that Audio Spectrogram

Key concepts

Multi-level VOTE Classification: This system classifies snoring sounds into different anatomical locations in the upper airway, such as Velum, Oropharynx, Tongue Base, and Epiglottis. The goal is to pinpoint exactly where the obstruction is occurring during sleep using acoustic signals from snoring.
Audio Spectrogram Transformer (AST): AST is a type of deep learning model that uses an attention mechanism to process audio signals by transforming them into spectrograms. It was found to be more effective than ResNet-50 at identifying obstructions at the Velum and Retropalatal levels.
Retropalatal (RP) vs. Retroglossal (RG): These are two specific regions identified in the study for classifying upper airway collapse. RP refers to an obstruction in the retropalatal area, while RG refers to an obstruction in the retroglossal area. The model was tested on identifying these distinct locations.
Full-Night Monitoring Potential: The study explored using a convolutional network to analyze continuous breathing sounds throughout a full night of sleep. This allowed researchers to track how airway obstruction sites change over time, showing that snoring often originates at the Velum and Oropharynx.

Terminology

AUC — 3× in this paper · explained in 2 episodes
ResNet-50 and Audio Spectrogram Transformer — 1× in this paper
Transformer — 1× in this paper · explained in 2 episodes

Summary

The gist The study investigated deep learning models to identify multi-level upper airway collapses in obstructive sleep apnea patients using snoring sounds, finding that Audio Spectrogram Transformer (AST) slightly outperformed ResNet-50 in identifying V and O level obstructions, while both models struggled with T, E, and RG classifications due to limited data.

Data Collection and Labeling

The researchers used snoring recordings from 37 subjects undergoing drug-induced sleep endoscopy (DISE) between 2020 and 2021 to train their models<ref:2408.16030#pg6>. Snoring sounds were labeled according to the VOTE (Velum, Orophar-ynx, Tongue Base, Epiglottis) classification system, resulting in specific counts for each class including 403 O and 77 T labels<ref:2408.16030#pg6>. The data preparation involved synchronizing endoscopic video tapes with breath sounds and aligning the snoring labels (Set A) and VOTE labels (Set B)<ref:2408.16030#pg9>. Furthermore, they created an additional label set where a clip involving any "V or O label was assigned an RP label, and a clip associated with any T or E" label was assigned an “RG” label<ref:2408.16030#pg9>. The signals were then truncated into 0.5-sec clips, keeping only those where the label occupied more than 50% of the duration of a 0.5-sec clip<ref:2408.16030#pg10>.

Model Architecture and Training

The study fine-tuned two models, ResNet-50 and Audio Spectrogram Transformer (AST), for a multi-label classification task<ref:2408.16030#pg10>. ResNet-50 is described as a classic deep neural network with a residual block that can pass information more efficiently from the upstream nodes to the downstream of the nets<ref:2408.16030#pg10>. AST adopted an attention mechanism and an encoder-decoder scheme to process the acoustic signals based on their temporal-spectral representations (spectrogram)<ref:2408.16030#pg10>. The input for these models was a log mel-spectrogram transformed using specific parameters such as window length=200, hop length=100, n fft=1024, and preemp=0.97<ref:2408.16030#pg11>. Training was conducted on a local computer using the GPU card NVIDIA GeForce RTX 3080 Ti with CUDA-11.7 and cuDNN-8.4.1<ref:2408.16030#pg11>. To manage data imbalance, class weights were adjusted, setting the T and E class weights to 5 while V, O, and N class weights remained at 1<ref:2408.16030#pg11>. Focal loss cross entropy was employed as the cost function with α set to 0.25 and γ set to 2<ref:2408.16030#pg11>.

Model Performance Evaluation

The performance of the ResNet-50 and AST models was evaluated using F1-score, Matthew’s correlation coefficient (MCC), and Area Under the Receiver Operating Characteristic curve (AUC)<ref:2408.16030#pg12>. For VOTE classification, the AST model demonstrated better performance than ResNet-50 in identifying snoring sounds generated at the V level (F1-score: 0.71 vs 0.64, MCC: 0.61 vs 0.49, and AUC: 0.89 vs 0.86)<ref:2408.16030#pg15>. Similarly, the AST model showed superior ability in identifying snoring sounds originating from the RP region (F1-score: 0.86 vs 0.84, MCC: 0.77 vs 0.73, and AUC: 0.97 vs 0.95)<ref:2408.16030#pg15>. However, both fine-tuned models failed to identify the snoring sounds at the T and E levels and those in the RG space<ref:2408.16030#pg17>.

Full-Night Monitoring Potential

The study also explored the potential for longitudinal tracking of airway collapse dynamics by retrospectively analyzing a full-night recording<ref:2408.16030#pg16>. This involved identifying inhalation, exhalation, and continuous adventitious sound episodes using a previously proposed convolutional bidirectional gated recurrent network model<ref:2408.16030#pg16>. The occupation rate of snoring was calculated assuming continuous adventitious sounds were all caused by snoring<ref:2408.16030#pg16>. Segments of continuous adventitious sound signals were fed into the multilabel VOTE classification model to identify the site of airway obstruction<ref:2408.16030#pg16>. Analysis showed that the model predicted that the snoring sounds mainly originated at the V level, followed by the O level, with a slight rise in E level association in the last half hour of sleep<ref:2408.16030#pg18>. This capability suggests that clinicians could adjust time slices to fit regions of research interest and synchronize results with clinical parameters like PSG data<ref:2408.16030#pg18>.

Limitations and Future Directions

The study acknowledged several limitations, including the acoustic features obtained during DISE may not fully replicate those produced during natural sleep, as induced snoring sounds tend to have a higher frequency component than natural snoring patterns<ref:2408.16030#pg17>. The endoscope might interfere with the airway structure and alter the acoustic patterns<ref:2408.16030#pg17>. Furthermore, assigning multi-level VOTE labels to each snore event by reviewing a DISE video is difficult, and the study did not differentiate between types of obstruction or consider obstruction severity<ref:2408.16030#pg17>. Future research should prioritize expanding the snoring sound dataset with VOTE labels and increasing its variety<ref:2408.16030#pg17>. The generalizability of the airway collapse monitor system to different patient populations and sleep environments needs further investigation<ref:2408.16030#pg17>. This model can be deployed into a smartphone app, allowing users to record sounds at home for analysis, facilitating communication with healthcare providers<ref:2408.16030#pg18>.

How it works

The core methodology involves training deep learning models on snoring sound clips derived from drug-induced sleep endoscopy (DISE) recordings<ref:2408.16030#pg6>. The input data consists of 0.5-second clips where the label occupies more than 50% of the clip's duration, derived from aligning endoscopic video tapes with breath sounds and VOTE labels<ref:2408.16030#pg10>. The models were fine-tuned for two multi-label classification tasks: identifying obstructions at VOTE levels (V, O, T, E) and identifying retropalatal (RP) and retroglossal (RG) obstructions<ref:2408.16030#pg10>. The input to the networks was a log mel-spectrogram transformed with specific parameters like window length=200 and hop length=100<ref:2408.16030#pg11>.

Model Comparison

The performance comparison between ResNet-50 and AST models showed that AST demonstrated better performance than ResNet-50 in identifying snoring sounds associated with the V level (F1-score: 0.71 vs 0.64, MCC: 0.61 vs 0.49, and AUC: 0.89 vs 0.86)<ref:2408.16030#pg15>. Similarly, AST performed slightly better than ResNet-50 in identifying snoring sounds originating from the RP region (F1-score: 0.86 vs 0.84, MCC: 0.77 vs 0.73, and AUC: 0.97 vs 0.95)<ref:2408.16030#pg15>. Both models showed limited success in identifying snoring sounds at the T and E levels and those in the RG space<ref:2408.16030#pg17>.

Longitudinal Analysis

The researchers demonstrated the potential for longitudinal tracking by recording tracheal sound during full-night sleep and retrospectively analyzing it using a convolutional bidirectional gated recurrent network model<ref:2408.16030#pg16>. This analysis identified episodes of apnea (no inhalation ≥10 seconds) and calculated the occupation rate of snoring<ref:2408.16030#pg16>. The resulting segments were then fed into the multilabel VOTE classification model to identify obstruction sites over time, revealing that snoring sounds mainly originated at the V level, followed by the O level<ref:2408.16030#pg18>.

Clinical Utility

The research suggests that clinical professionals can relate a snoring sound’s origin to T and E levels or the RG space if models indicate no signs of involvement at V and O levels in the RP space<ref:2408.16030#pg17>. The system has potential for real-time use, such as immediately knowing whether a mandibular advance device effectively changes airway collapse dynamics during a patient's sleep<ref:2408.16030#pg18>. Moreover, the models can be deployed into smartphone apps for at-home recording and consultation with healthcare providers<ref:2408.16030#pg18>. This objective information can help clinicians improve diagnostic accuracy, personalize treatment plans, monitor treatment response, and enhance patient care<ref:2408.16030#pg18>.

Improvements for AI systems

Improve model performance on T and E level classifications by addressing data imbalance through more sophisticated loss functions beyond Focal loss cross entropy. The paper notes that The limited samples made the trained models poorly fitted for the test data, suggesting that further exploration of class weighting or synthetic data generation specifically for these underrepresented classes could yield better results.
Enhance model robustness against acoustic variations by incorporating features from different acoustic domains in addition to log mel-spectrograms. The paper mentions that The acoustic features of snoring sounds obtained during DISE may not fully replicate those produced during natural sleep, suggesting the model could benefit from integrating features derived from other signal processing techniques mentioned in related work, such as wavelet-based or texture features.
Implement a dynamic time-slice adjustment mechanism synchronized with external clinical parameters for real-time monitoring. The paper suggests that the user can more specifically adjust the time slice length and location to fit a region of research interest if the analysis result is synchronized with other clinical parameters, such as PSG data, allowing the AI system to provide contextually relevant insights during sleep.
Develop a multi-level classification system capable of differentiating between different types of collapse based on acoustic signatures. Future research should prioritize The association between snoring acoustics and the types (anteroposterior, laterolateral, and concentric) and extent (partial or complete) of airway obstruction, enabling the AI to classify not just location but also the nature of the obstruction.
Create a portable smartphone application for at-home snoring sound analysis. The paper proposes that A snorer can easily use the phone to record the snoring sounds during sleep at home and consult the app for the analysis results, facilitating communication with healthcare providers, turning a diagnostic tool into an accessible patient monitoring device.

Abstract

Objective This study evaluated deep learning models, specifically ResNet-50 and Audio Spectrogram Transformer (AST), to identify multi-level upper airway collapses in obstructive sleep apnea (OSA) patients using snoring sounds. Methods Snoring recordings from 37 subjects undergoing drug-induced sleep endoscopy (DISE) between 2020 and 2021 were used to train models for two multi-label tasks: identifying obstructions at the Velum (V), Oropharynx (O), Tongue-base (T), and Epiglottis (E) levels, and the retropalatal (RP) vs. retroglossal (RG) levels. Results The AST model significantly outperformed ResNet-50 (P <0.05 after Bonferroni correction), effectively identifying V-, O-, and RP-level obstructions (F1-score: 0.59-0.87, Matthews Correlation Coefficient: 0.50-0.80, and AUC: 0.87-0.97). However, the performance of both models at the T, E, and RG levels remained limited due to small sample sizes, even with the Synthetic Minority Over-sampling Technique. A full-night recording analysis further demonstrated the feasibility of acoustic profiling for airway dynamics. Conclusion The AST models can effectively identify airway obstruction at V, O, and RP levels via snoring sounds. The proposed models offer a promising non-invasive tool for tracking overnight airway dynamics and assisting in clinical treatment planning.