Evidence for Phenotype-Driven Disparities in Freezing of Gait Detection and Approaches to Bias Mitigation

arXiv:2502.09626 · eess.SP, cs.LG · Submitted 2025-01-29 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Hi, I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: We're looking at the paper: "Evidence for Phenotype-Driven Disparities in Freezing of Gait Detection and Approaches to Bias Mitigation".

Jane: Here's the gist of the paper. The gist The systematic evaluation of bias and fairness in state-of-the-art human activity recognition models for freezing of gait detection revealed that conventional mitigation techniques were often ineffective, while transfer learning approaches showed the most promise for reducing disparities across FOG phenotypes and demographics.

Tom: Freezing of gait (FOG) is a debilitating symptom of Parkinson’s disease (PD) that is challenging for clinicians to capture and evaluate during time-limited clinical assessments.

Jane: This phenotype-specific bias is particularly concerning as tremulous and akinetic FOG may require different clinical management.

Tom: Few of these models have been comprehensively evaluated for performance across phenotypes, which could potentially introduce systematic error.

Title and authors: Tom: Now they test four different ways to try and fix this problem. These include conventional methods like threshold optimization we talked about, adversarial debiasing which tries to make the features attribute-agnostic >

Lu: And then they introduce two transfer learning based approaches. One is multi-site transfer from datasets that have more diverse examples, and the other is fine-tuning large pretrained foundation models >

Meng: I’m curious about the cost here. Transfer learning sounds promising because it exposes the model to more data, but what does that actually do for us when we're running this on a live system >

Jane: Well, they show that transfer learning from multi-site datasets was the most effective approach overall in terms of both fairness and performance improvements across those models >

Tom: They found multi-site transfer increased the F1-score by zero point zero two four for Model A2 and plus zero point zero one seven for Model A3, and it improved demographic parity by zero point zero three seven >

Lu: And that same transfer learning also improved equalized odds by zero point zero four five for both architectures, which is a solid improvement over the baseline > <ref:2502.09626#pg2>

Meng: So, it’s not just about getting a higher score; it's about seeing these specific metrics move in the right direction across those protected groups >

Lalam: They did show one experiment where transfer learning from Model B2 on one dataset actually managed to reduce bias down to meet that four-fifths rule threshold, but they noted it came with a substantial performance cost >

Jane: So, the message here is that transferring knowledge across more diverse sites is a much better path than just trying different local tuning methods >

Tom: It really shows that the way we train these models matters more than just picking the right hyperparameter for a fixed architecture >

The paper's summary: Tom: The main improvement they suggest is shifting focus from just accuracy to systematically assessing differential performance across different FOG manifestations >

Lu: They argue that best practices for evaluating this differential performance haven't really been established, and they point out a gap where prior studies have shown higher performance for tremulous FOG compared to akinetic FOG with an intraclass correlation coefficient of roughly zero point eight six versus zero point seven eight > <ref:2502.09626#pg3>

Meng: So, they are calling out that the existing benchmarks aren't looking at this phenotype stratification properly, which is a huge issue if we want reliable clinical tools >

Jane: They suggest that since tremulous and akinetic episodes co-occur in over eighty percent of patients, systematically evaluating performance differences across those two manifestations is really important > <ref:2502.09626#pg3>

Lalam: The paper concludes that achieving equitable FOG detection requires proactive considerations during data collection, specifically stratified sampling by phenotype and demographics, rather than just relying on downstream bias mitigation >

Tom: That’s a strong stance. They aren't just saying "mitigate the bias after training," they are saying we need to change how we collect the data in the first place >

Lu: They suggest that future efforts might require intentional sampling strategies that explicitly enrich underrepresented FOG phenotypes and demographic subgroups during data collection >

Meng: I worry about implementing that. Stratified sampling by phenotype sounds incredibly labor-intensive, and it adds complexity to any production pipeline we have to manage >

Jane: But they argue that the performance differences arise from genuine pathophysiological differences in motor mechanisms between those phenotypes, so you can't just ignore them >

The paper's improvements: Tom: So, to wrap up the Evidence for Phenotype-Driven Disparities in Freezing of Gait Detection and Approaches to Bias Mitigation paper: the key finding is that transfer learning from multi-site datasets was substantially better than conventional mitigation methods at reducing disparities >

Lu: And they emphasize that for developing equitable HAR models that reliably detect FOG across all patient phenotypes, transfer learning across diverse datasets is essential >

Meng: From a practical side, it means we need to look beyond just the model weights and think about the data pipeline itself if we want something that actually works reliably in the real world >

Jane: It seems like they’re pushing for a proactive approach where you sample specifically for those underrepresented groups instead of just patching problems later >

Lalam: They really put it on data collection. The paper is about moving toward intentional sampling strategies to enrich those FOG phenotypes and demographic subgroups during data collection >

Tom: So, we're done with this one. It’s clear that for FOG detection, the path forward involves looking at how we collect the data rather than just tweaking the model after it’s been built >

Jane: We’ll see what other papers come out next that build on these ideas >

Conclusion: Tom: So to wrap up, the paper "Evidence for Phenotype-Driven Disparities in Freezing of Gait Detection and Approaches to Bias Mitigation" shows that standard models are hitting a wall when it comes to fairness across different FOG phenotypes and demographics >

Jane: Right. The main thing is that the performance improvements we see in these state-of-the-art human activity recognition models don't actually translate into better fairness metrics like demographic parity or equalized odds >

Lu: Exactly. The authors systematically evaluated five different AI models and found that increasing architectural complexity didn't fix the bias; it actually made things worse for fairness metrics >

Meng: From a deployment standpoint, if you’re looking at those baseline performance gains without any fairness fixes, you’re just baking systematic errors into your production system >

Lalam: The paper strongly points to transfer learning as the most promising mitigation strategy because multi-site transfer increased F1-scores by zero point zero two four for Model A2 and improved demographic parity by zero point zero three seven >

Tom: So it’s not just about finding a better threshold; it's about moving the model to see more diverse examples before you even start tuning parameters >

Jane: And they found that transfer learning from multi-site datasets was the most effective approach overall in terms of both fairness and performance improvements across those models >

Lu: They showed that this transfer learning actually helped reduce disparities, and one experiment with Model B2 on one dataset managed to hit the four-fifths rule threshold, but it came at a substantial cost to its accuracy >

Meng: That performance cost is something I’d have to worry about before I deploy anything, especially when dealing with patient data where every missed detection matters >

Lalam: They also highlighted that FOG phenotype itself is a major source of bias because akinetic episodes are way underrepresented in the datasets and that’s what causes the performance disparities >

Tom: So they’re calling for intentional sampling strategies during data collection, specifically enriching those underrepresented FOG phenotypes and demographic subgroups >

Jane: That means we need to stop just patching the model after training and start thinking about how we collect our data from the beginning >

Lu: They suggest that a future direction might be developing phenotype-specific models where you train separate models for distinct FOG manifestations rather than one general system >

Meng: Stratified sampling sounds incredibly labor-intensive, but if it yields a truly robust and safe model, then maybe we’ll have to consider it for high-stakes applications >

Lalam: So that’s the gist of "Evidence for Phenotype-Driven Disparities in Freezing of Gait Detection and Approaches to Bias Mitigation" – the takeaway is proactive data collection over reactive mitigation >

Tom: Yeah, it highlights that just achieving high accuracy isn't enough if you don't control for those underlying clinical differences >

Jane: It really makes you think about how we build these systems for people who actually have these conditions and need reliable detection across all of them >

Lu: We’ll look at how other papers are tackling domain gaps in the next segment.

Timothy Odonga, Christine D. Esper, Stewart A. Factor, J Lucas McKay, Hyeokhyen Kwon

Emory University · Georgia Institute of Technology

eess.SP, cs.LG

Submitted: 2025-01-29

Updated: 2026-10-16

Comments: Revised manuscript for EJN Special Issue Submission

License: http://creativecommons.org/licenses/by-nc-sa/4.0/

Importance score: 84/100

The gist: The gist The systematic evaluation of bias and fairness in state-of-the-art human activity recognition models for freezing of gait detection revealed that conventional mitigation techniques were

Key concepts

Freezing of Gait (FOG) Phenotypes: These are different types of freezing episodes associated with Parkinson's disease, such as tremulous or akinetic FOG. The paper found that certain phenotypes are significantly underrepresented in existing datasets, leading to performance disparities when models are trained on them.
Group Fairness Metrics: These metrics quantify fairness by measuring model performance across different groups defined by protected attributes like phenotype, age, and sex. Key measures include Demographic Parity (equal positive predictions) and Equalized Odds (equal true positive/false positive rates).
Transfer Learning: This involves taking a model trained on one set of data and adapting it to a new task or dataset. The study found that using transfer learning from multi-site datasets significantly improved fairness metrics, suggesting exposure to diverse data helps create more robust and equitable models.
Intentional Sampling Strategies: Instead of just fixing errors after training, this means actively collecting more data points specifically from underrepresented groups, such as rare FOG phenotypes or specific age/sex combinations. This proactive step is crucial because the performance gaps often stem from genuine differences in how different phenotypes move.

Terminology

Transfer Learning — 3× in this paper · explained in 16 episodes
Adversarial Debiasing — 2× in this paper
Fine-Tuning — 1× in this paper · explained in 5 episodes

Summary

The gist The systematic evaluation of bias and fairness in state-of-the-art human activity recognition models for freezing of gait detection revealed that conventional mitigation techniques were often ineffective, while transfer learning approaches showed the most promise for reducing disparities across FOG phenotypes and demographics.

Introduction to the Problem

Freezing of gait (FOG) is a debilitating symptom of Parkinson’s disease (PD) that is challenging for clinicians to capture and evaluate during time-limited clinical assessments. This phenotype-specific bias is particularly concerning as tremulous and akinetic FOG may require different clinical management. Few of these models have been comprehensively evaluated for performance across phenotypes, which could potentially introduce systematic error.

Defining Bias and Fairness Metrics

The study defined protected attributes as clinical attributes such as FOG phenotype and disease duration, as well as demographic characteristics such as age and sex. Bias is defined as systematic errors leading to unequal outcomes. Fairness was adopted at the group level, where fairness is satisfied when average model performance is consistent across groups defined by protected attributes. The study quantified fairness using three notions of group fairness: Demographic parity, which requires equal proportions of positive predictions between protected groups. Equalized odds require equal true positive and false positive rates across subject groups conditional on the true outcome.

Model Evaluation and Baseline Performance

Five state-of-the-art human activity recognition (HAR) models were systematically evaluated for their bias and fairness across FOG phenotypes and demographic strata. Model performance improved with architectural complexity across the multi-site datasets, as shown in Table 6 and Figure 2. Despite these performance gains, all models exhibited suboptimal fairness across clinical and demographic attributes, with no model achieving the 0.8 four-fifths rule threshold for any fairness metric. Critically, increased model performance did not correspond to improved fairness, suggesting that standard training procedures introduced systematic biases that require targeted mitigation.

Bias Mitigation Approaches

Four bias mitigation approaches were assessed: conventional methods (threshold optimization and adversarial debiasing) and two transfer learning-based approaches (multi-site transfer and fine-tuning large pretrained models). Threshold optimization was applied to Model A1 to reduce bias quantified by DPR, EOR, and EOD between the protected groups. Adversarial debiasing aimed to train attribute-agnostic feature representations in the implemented FOG detection models. Transfer learning from multi-site datasets was hypothesized to improve fairness and model performance by exposing models to more diverse training examples. Fine-tuning large-scale pretrained models (foundation models) was investigated as a fourth approach to reduce bias.

Key Findings and Conclusion

Transfer learning approaches substantially outperformed other mitigation strategies in both fairness gains and performance improvements. Multi-site transfer learning increased F1-scores by +0.024 (Model A2) and +0.017 (Model A3), while improving DPR (+0.037) and EOR (+0.045) for both architectures. The only experiment conducted in which bias mitigation reduced bias to meet the four-fifths rule was transfer learning from Model B2 on one dataset, but at a substantial performance cost. The FOG phenotype emerged as a significant source of bias, with akinetic episodes substantially underrepresented in datasets and associated with performance disparities. Future efforts may require intentional sampling strategies that explicitly enrich underrepresented FOG phenotypes and demographic subgroups during data collection. The paper concludes that achieving equitable FOG detection requires proactive considerations during data collection, including stratified sampling by phenotype and demographics, rather than relying solely on bias mitigation approaches. The findings show that achieving equitable FOG detection requires proactive considerations during data collection, including stratified sampling by phenotype and demographics, rather than relying solely on bias mitigation approaches>.

Limitations

The foundation models were not necessarily trained on subjects with movement disorders, creating a domain gap between the pretraining data and the clinical FOG detection task. The study did not examine intersectional biases arising from combinations of protected attributes. Different window sizes may capture FOG phenotypes differently, with shorter windows potentially favoring brief akinetic episodes and longer windows favoring sustained tremulous freezing.

Future Work

Future work may require intentional sampling strategies that explicitly enrich underrepresented FOG phenotypes and demographic subgroups during data collection. An alternative strategy may be the development of phenotype-specific or stratified models, in which separate models are trained for distinct FOG manifestations <ref:2502.

Improvements for AI systems

Evaluate FOG detection models using group fairness metrics like Demographic Parity Ratio (DPR) and Equalized Odds Ratio (EOR) across protected attributes including FOG phenotype, age, sex, and disease duration. This allows for the systematic evaluation of bias as a systematic error that leads to unequal outcomes mentioned in the paper's definition of bias.
Implement transfer learning from multi-site datasets to improve fairness and performance, as this approach was shown to significantly reduce disparities across models compared to conventional mitigation methods. Specifically, this can help develop equitable HAR models that reliably detect FOG across all patient phenotypes.
Utilize foundation model feature transfer (Model B2) pretrained on demographically diverse data like NHANES for FOG detection, as it achieved the highest fairness metrics including DPR values exceeding 0.80 for all protected attributes. This system can potentially reduce bias in FOG detection models by leveraging inherited features learned during pretraining.
Employ multi-head adversarial debiasing to train attribute agnostic feature representations, which aims to create a classifier that performs FOG detection without relying on protected attributes. This helps ensure that the model's performance is not dependent on demographic or clinical characteristics.
Introduce intentional sampling strategies during data collection, specifically enriching underrepresented FOG phenotypes and demographic subgroups, instead of relying solely on downstream mitigation. This proactively addresses the finding that the performance differences arise from genuine pathophysiological differences in motor mechanisms between phenotypes.

Abstract

Freezing of gait (FOG) is a debilitating feature of Parkinson's disease (PD) and a common cause of injurious falls. Recent advances in wearable-based human activity recognition (HAR) have enabled FOG detection, but bias and fairness in these models remain understudied. Bias is defined as a systematic error leading to unequal outcomes, while fairness refers to consistent performance across subject groups. Biased models could systematically underserve patients with specific FOG phenotypes or demographics, widening care disparities. We evaluated the bias and fairness of state-of-the-art HAR models for FOG detection across FOG phenotypes and patient demographics using 4 public wearable datasets (145 subjects and 59.8 hours of recording) with multiple on-body sensor locations and temporal annotations on FOG episodes across various behavioral test protocols. Four mitigation approaches were assessed: conventional methods (threshold optimization and adversarial debiasing) and two transfer learning-based approaches (multi-site transfer and fine-tuning pre-trained models). Fairness was primarily quantified using the demographic parity ratio (DPR) and the equalized odds ratio (EOR). HAR models exhibited substantial bias (DPR & EOR < 0.8) across age, sex, disease duration, and critically, FOG phenotype. This phenotype-specific bias is particularly concerning as tremulous and akinetic FOG may require different clinical management. Conventional methods failed: threshold optimization (DPR=-0.126, EOR=+0.063) and adversarial debiasing (DPR=-0.008, EOR=-0.001). In contrast, multi-site transfer learning improved fairness (DPR=+0.037, p<0.01; EOR=+0.045, p<0.01) and performance (F1-score=+0.020, p<0.05), though neither metric reached 0.8. Transfer learning across diverse datasets may partially mitigate these disparities, helping wearable-based FOG monitoring benefit all individuals with PD.