UniPrompt-CL: Efficient Single-Inference Continual Learning for Medical Image Classification

summary

Video file (mp4)

The gist

The gist The proposed UniPrompt-CL method is a medical-oriented Prompt-based Continual Learning (PCL) approach that improves prompt pool design via a minimally expanding unified prompt pool and a new

In short

UniPrompt-CL is a medical image classification method that improves continual learning by using a single backbone and a unified prompt pool shared across all transformer layers. It introduces minimal prompt expansion and a consistency regularization term to achieve better stability and efficiency, resulting in improved accuracy on diabetic retinopathy while reducing forgetting.

Key concepts

Unified Prompt Pool
Instead of separate prompt pools for each layer, UniPrompt-CL shares one pool across all transformer layers. This maximizes the use of limited prompt capacity by ensuring that the same set of learned prompts is available at every part of the model, promoting efficient feature tracking throughout the backbone.
Minimal Prompt Expansion
When adapting to a new medical domain, UniPrompt-CL expands the prompt set by adding only 20% more prompts. This controlled expansion ensures that new information is captured without overwhelming the system or increasing computational cost unnecessarily, leading to better adaptation.
Consistency-Enforcing Regularization
A novel regularization term is added to the training loss. This term encourages stable yet adaptive learning, helping the model retain knowledge from previous stages while still being plastic enough to learn new domain features effectively, balancing stability and plasticity.

Terminology used across episodes

This episode discusses

The paper

UniPrompt-CL: Efficient Single-Inference Continual Learning for Medical Image Classification · Read on arXiv

Gyutae Oh, Jitae Shin

Department of Electrical and Computer Engineering, Sungkyunkwan University

Medical AI systems must remain reliable as data distributions shift across hospitals, imaging devices, and patient populations, yet privacy constraints often prevent storing or revisiting past patient data. We present UniPrompt-CL, a rehearsal-free domain-incremental learning framework for medical image classification using a single backbone and a single inference. Rather than relying on multiple backbones, repeated ViT inferences, or independent layer-wise prompt pools, UniPrompt-CL shares a unified prompt pool across transformer layers and expands it with only a few prompts per new domain. This reduces cross-layer redundancy while enabling efficient adaptation. On diabetic retinopathy, UniPrompt-CL improves average accuracy by 2.0 percentage points and substantially reduces forgetting relative to the strongest backbone-matched baseline. On skin cancer, it achieves competitive accuracy at the lowest training cost among DINOv2-B configurations. A controlled comparison varying only pool organization shows that the benefit arises from retention rather than learning each new domain: it is largest on the medical benchmarks, smaller on OfficeHome, and reverses on a heterogeneous six-stage DomainNet stream. These results suggest that cross-layer prompt sharing with lightweight expansion is practical for medical models requiring repeated retraining under limited on-site compute, while identifying the domain shifts under which it helps.

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "UniPrompt-CL: Efficient Single-Inference Continual Learning for Medical Image Classification".

Jane: Here's the gist of the paper. The gist The proposed UniPrompt-CL method is a medical-oriented Prompt-based Continual Learning (PCL) approach that improves prompt pool design via a minimally expanding unified prompt pool and a new regularization term, achieving better stability–plasticity trade-off with lower computational cost<ref:2508.10954#pg14>.

Tom: Prior PCL methods often underperform or fail to transfer to medical data due to domain bias, institutional constraints, and subtle inter-stage boundaries<ref:2508.10954#pg3>.

Jane: Medical images require different promptlearning strategies than natural images<ref:2508.10954#pg4>.

Tom: Existing PCL methods developed for natural images often learn wide prompt regions to accommodate broad feature differences, which can be inefficient for medical data where fundamental anatomical structures remain consistently important across layers and tasks<ref:2508.10954#pg5>.

Paper summary: Jane: So looking at this paper, "UniPrompt-CL: Efficient Single-Inference Continual Learning for Medical Image Classification," the authors are proposing a specific way to manage those prompts inside a medical AI model to make continual learning more stable and efficient than what's currently available.

Tom: The title itself points directly to the core idea: using a single inference pass for continual learning in medical image classification, and they claim this method improves accuracy by one to three percentage points over strong baselines.

Lu: It’s about how they design the prompt pool—by unifying it across layers and only expanding it minimally, plus using that new regularization term Ls to guide the learning process toward better stability-plasticity trade-offs.

Meng: For someone deploying this to a real system, the value is in that you get high performance while keeping the inference cost low because you're not running multiple passes or complex pipelines.

Jane: The implication here is that for medical AI where data distribution shifts frequently but compute resources are constrained, this approach offers a practical way to mitigate catastrophic forgetting while maintaining strong feature tracking across the model layers.

Tom: It really boils down to how they managed the prompt design—the unified pool and minimal expansion—which seems to be the key mechanism allowing them to achieve better retention of prior knowledge while still capturing fine-grained features for new medical tasks.

Conclusion: Tom: So we've seen how UniPrompt-CL uses a single inference pass to handle continual learning in medical image classification, and now we gotta talk about what that actually means for us.

Jane: It boils down to this paper, "UniPrompt-CL: Efficient Single-Inference Continual Learning for Medical Image Classification," written by some folks. The main thing is they tackle the problem of keeping an AI model accurate as it sees new medical data without having to retrain everything from scratch every time.

Tom: Yeah, so they used a unified prompt pool and this minimal expansion strategy to manage the prompts across layers, and that's what keeps things stable. But why does that matter for a doctor or a hospital?

Jane: Because in medicine, you can't just throw all your past patient data into one giant pile anymore because of privacy rules. This method lets the AI adapt to new hospitals or new scanner types while still remembering what it learned before without forgetting the old stuff.

Tom: Exactly. It’s about efficiency here too—they use just one backbone and one inference pass, which is way faster than running a whole pipeline every time you want to update something. Meng, from a deployment standpoint, how does that single pass translate to actual running costs?

Meng: Well, it means less compute needed per update cycle. If I'm shipping this to a clinic where the machine has limited GPU power for retraining, keeping the complexity low with just one pass is actually pretty good for deployment. But we still have to worry about how those prompts are organized across different medical domains.

Jane: That’s a fair point about the cost, Meng. The paper shows they found that this prompt design helps them keep accuracy up—improving it by one to three percentage points on tests like diabetic retinopathy without needing massive training overhead.

Tom: So, the main implication is that for medical AI, we don't have to sacrifice stability just for a little bit of extra accuracy or speed in every single update. It seems the prompt structure itself is smarter than just throwing more layers or more backbones at it.

Jane: It really suggests that when you design these models, especially for specialized fields like medicine, how you manage those internal connections—those prompts—is just as important as the size of the backbone itself.

Tom: Right. So if this prompt strategy works so well here, what’s next? Are they just going to use this for skin cancer and diabetic retinopathy?

Jane: They're looking to see if it holds up on more complex things, like different types of imaging modalities—maybe moving from just standard images to things like CT or MRI scans. We gotta keep asking how robust these prompt designs are across truly different data types.

More episodes

← Home