Rethinking Synthetic Rationale Supervision for Disease Prediction

summary

Video file (mp4)

The gist

The gist: Supervised fine-tuning with synthetic rationale data consistently and substantially hurts prediction performance relative to label-only fine-tuning for five-year Alzheimer’s disease and

In short

The study tested using synthetic rationales to improve Alzheimer's disease prediction by teaching models 'why' they predict a certain outcome. The results showed that training models on these generated rationales consistently and substantially hurt their predictive performance compared to training with only the final labels. This suggests that conditioning predictions on synthetic explanations degrades accuracy.

Key concepts

Rationale-based SFT
This technique involves fine-tuning a language model not just to predict an answer, but also to generate a detailed explanation or 'why' behind that prediction. The researchers tested if teaching the model this reasoning process improved its ability to accurately predict diseases like Alzheimer's.
Label-Only Fine-tuning
This is the baseline method where a model is trained exclusively on the correct answers (labels) without any additional explanations or rationales. The study found that this simpler approach actually resulted in better predictive performance than methods involving synthetic rationales.
Structural Conflict
This refers to a conflict between two goals: creating a medically plausible story (a good rationale) and learning the specific patterns needed to distinguish future cases from controls (discriminative optimization). When training on rationales, the model prioritized creating a story over learning the precise features that matter for accurate prediction.

Terminology used across episodes

This episode discusses

The paper

Rethinking Synthetic Rationale Supervision for Disease Prediction · Read on arXiv

Buxin Su, Bingxuan Li, Cheng Qian, Yiwei Wang, Jin Jin, Bingxin Zhao

University of Pennsylvania · University of Illinois Urbana-Champaign · University of California, Merced

Supervised fine-tuning with synthetic rationales is widely assumed to improve clinical prediction by teaching models not only what to predict but why. We test this assumption on five-year Alzheimer's disease and related dementias (ADRD) prediction from longitudinal health histories. In this setting, we generate training rationales using the correct labels, then fine-tune models to output a rationale followed by a label. Across a large-scale controlled experiment of 504 configurations, we find that Rationale First SFT consistently and substantially hurts predictive performance relative to Label Only fine-tuning. The degradation persists across model families and data scales, and is not resolved by using a reasoning-oriented base model. A fixed-protocol cardiovascular disease (CVD) validation reproduces similar findings. Crucially, the failure is not explained by poor rationale quality: human expert annotation confirms that the generated rationales are medically accurate and faithfully grounded in patient-specific evidence, and few-shot experiments show that the same rationales improve performance when used as inference-time demonstrations rather than training targets. A simple label-ordering intervention clarifies the mechanism: placing the prediction before the same rationale restores predictive performance. This suggests that conditioning predictions on generated rationales contributes to the degradation. We hope our work paves the way toward a more precise understanding of when and how rationale-based supervision helps and when it does not, guiding the responsible development of language models for high-stakes clinical prediction.

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "Rethinking Synthetic Rationale Supervision for Disease Prediction".

Jane: Here's the gist of the paper. The gist: Supervised fine-tuning with synthetic rationale data consistently and substantially hurts prediction performance relative to label-only fine-tuning for five-year Alzheimer’s disease and related dementias prediction (Page 1).

Tom: How it works Supervised fine-tuning with synthetic rationale data is widely assumed to improve language model performance on clinical prediction tasks by teaching models not just what to predict but why (Page 1).

Jane: This assumption was tested on five-year Alzheimer’s disease and related dementias (ADRD) prediction from longitudinal health histories across a large-scale controlled experiment of 504 configurations (Page 1).

Tom: The researchers found that rationale-based SFT consistently and substantially hurts prediction performance relative to label-only fine-tuning (Page 1).

Paper summary: Tom: So, looking at this paper, "Rethinking Synthetic Rationale Supervision for Disease Prediction," what’s the final verdict on using these synthetic rationales as a training target?

Jane: The authors conclude that rationale-based supervised fine-tuning appears to shift the optimization process away from learning the discriminative boundary between cases and controls in this setting. It suggests that label-only fine-tuning is superior for five-year ADRD prediction on longitudinal health histories.

Lu: It’s a clear finding: for this type of clinical prediction task, the mechanism seems to favor direct supervision over synthetic explanation generation during training.

Meng: And the limitation they mention is that their work is bounded by its focus on one disease and cohort, as well as using LLM-generated rationales instead of those written by human clinicians. That means we can’t just take this result and apply it blindly to any high-stakes domain without checking those assumptions.

Jane: Exactly. So the paper warns us that generating synthetic explanations isn't automatically a shortcut for trustworthy explanations in high-stakes domains, because the generated rationale is unreliable for discriminative informativeness.

Tom: It’s about understanding when and how rationale supervision actually helps, and more importantly, when it actually hurts performance on these kinds of tasks. That distinction is crucial for responsible development of language models in clinical prediction.

Conclusion: Tom: So, to wrap up, this paper is about supervised fine-tuning using synthetic rationales and why that method actually hurts prediction performance for Alzheimer's disease stuff on longitudinal health data.

Jane: It’s interesting how they tested the idea that teaching a model *why* it predicts something might help it predict better things in the real world.

Lu: The core finding is pretty clear, though: using those synthetic rationales across fifty-four different configurations consistently makes models trained with just labels do better on ROC-AUC scores.

Meng: So, the optimization isn't actually learning a better boundary; it’s getting distracted by making the explanation sound plausible.

Tom: That’s what I mean. It seems like the model starts trying to reproduce a story instead of just finding the right data points that separate cases from controls in that specific dataset.

Jane: But wait, they did check if the quality of those synthetic rationales was bad, and it turns out human experts say they are actually medically accurate and grounded in patient evidence.

Lu: Yeah, because the failure isn't about the rationale being wrong; it’s about a structural conflict between making a smooth story and actually learning what makes one case different from another.

Tom: So, the implication is that for these kinds of high-stakes clinical predictions, sticking to label-only fine-tuning keeps the model focused on what matters.

Jane: That means we shouldn't just toss synthetic explanations into a training pipeline hoping they’ll make things better; we need a different approach entirely.

Lu: Exactly. The paper points toward figuring out precisely when conditioning predictions on generated rationales actually helps versus when it just adds noise to the optimization budget.

Meng: It suggests that for us deploying models, we need to be careful about what kind of supervision we feed them if they are going to be making decisions in a clinical setting.

Tom: So the next step is figuring out how we can use those rationales without it hurting our actual prediction accuracy.

More episodes

← Home