Representation Drift in Continual Learning: LoRA versus N-gram Memory

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: Today's paper: "Representation Drift in Continual Learning".

Jane: The gist The base stays fixed and every change is local and reversible,

Tom: First, who's behind it and why it matters.

Paper summary: Tom: So we're looking at this paper called "Representation Drift in Continual Learning: LoRA versus N-gram Memory." It's about how a language model that keeps learning online doesn't slowly lose its basic abilities. The authors are asking which part of the model should absorb new knowledge when it updates continually by measuring how each choice changes the model's internal representations.

Jane: That sounds really important because we all use these models more and more, and if they start forgetting things, that could cause big problems for how we build and use AI systems. So what exactly are they testing here?

Tom: They set up a forty-eight-layer mixture-of-experts model with a native pretrained N-gram memory <ref:2610.14921#pg1,a 48-layer mixture-of-experts model with a native pretrained N>. They then train seven different tasks using three different update mechanisms. Those mechanisms are updating only the N-gram memory table, continually training one shared LoRA adapter, and training a fresh LoRA per task that gets merged into the backbone.

Lu: The core idea here is to capture fixed-context representations at every layer and every task boundary. They measure these representations against two references: general math and code inputs for big-K drift, and learned task inputs for small-k drift (<ref:2610.14921#pg2>).

Meng: When I think about this from a deployment side, I'm thinking about what kind of memory mechanism actually scales well. The authors are comparing the N-gram approach against two LoRA strategies under matched data exposure to see where new knowledge should be written for an AI that keeps learning online.

Lalam: My literature sweep points toward representation drift as a key concept, and this paper directly addresses how that drift manifests in different update strategies. The paper claims that updating only the N-gram memory changes the base model’s representations of general math and code inputs by twenty-six percent/fifteen percent relative RMS, against sixty-four percent/thirty-five percent for merged LoRA and ninety-seven percent/thirty-nine percent for shared LoRA (<ref:2610.14921#pg1>).

Jane: Those numbers are what grab my attention right now. So, the paper suggests that updating just the N-gram memory causes a big-K drift of twenty-six percent to fifteen percent on math and code inputs, which is much smaller than the drift seen with LoRA, which is sixty-four percent to ninety-seven percent <ref:2610.14921#pg1>. What does that actually mean for us?

Tom: It means that updating only the N-gram memory leaves the base model's representations of general math and code inputs nearly untouched. And this drift doesn't grow with the number of tasks; it reaches twenty-five percent after just the first task and stays there for all seven (<ref:2610.14921#pg1>).

Lu: That finding is interesting because generic behavior stays essentially unchanged, meaning no code problem gets lost, which is something we really want. But they also found that the per-round changes cancel out in the N-gram case, with a realized squared drift being fifteen percent of the sum of per-round squared changes, compared to forty percent to fifty-six percent for LoRA (<ref:2610.14921#pg2>).

Paper summary: Meng: From an engineering standpoint, that cancellation sounds good for stability, but they also point out a cost: the cost is interference among stored tasks rather than damage to the base model itself because it acquires difficult tasks more weakly and its stored tasks interfere (<ref:2610.14921#pg3>).

Jane: That's a subtle distinction. So, for N-gram memory, the interference happens between existing stored tasks, not necessarily eroding the general knowledge base as much as those LoRA methods do. But if it does interfere, how does that look in terms of representation drift?

Tom: They define representation drift as key drift Kt minus Ki at each layer. The paper shows that every output change comes from this key drift, and in particular at the head (<ref:2610.14921#pg3>). Small-ki drift compares each learned task’s inputs with the checkpoint at which that task had just been learned, measuring what happens to acquired knowledge afterwards.

Lalam: The paper connects this small-ki drift directly to retention loss through a bound: ∆NLLt i is less than or equal to sqrt(two) times the norm of U times the norm of e t i (<ref:2610.14921#pg3>). This means that even with small key drift, there's still a connection between that drift and how much knowledge you lose when you add new things.

Lu: And they show that for N-gram memory, an input whose addressed rows were never rewritten is exactly unchanged, which is why generic drift stays small (<ref:2610.14921#pg3>). But old-task inputs share rows with new tasks, leading to large small-ki drift because those old inputs share rows with later tasks that rewrite them (<ref:2610.14921#pg8>).

Meng: So for a system running on real traffic, that means the performance might degrade specifically for the data patterns that overlap between old and new tasks, even if the overall math capabilities look fine. That sounds like a specific failure mode we'd have to debug at scale.

Jane: What does that tell us about the architecture? The paper says N-gram memory has one source term, which is the injection change ∆J(x) of Equation (twenty-eight) carried through the frozen suffix (<ref:2610.14921#pg8>). If ∆T R(x) is zero on the full causal prefix of an old input, its entire forward pass is unchanged (<ref:2610.14921#pg8>).

Tom: That single source term idea simplifies things a lot because it means we can track the drift down to that injection change. It's small on generic inputs—the stuff that doesn't overlap with new tasks—but large on old-task inputs which read rows later tasks rewrite (<ref:2610.14921#pg8>).

Lu: The layer profile they show is pretty telling, showing that on old-task inputs, the N-gram drift reaches thirty-one percent within seven layers of reading, while generic inputs at the same depth only move by four percent (<ref:2610.14921#pg8>). That's a significant difference in how sensitive the model is to new knowledge depending on what it's processing.

Meng: So, if we were building a system, that suggests we might need to be very careful about how we structure our memory retrieval or update mechanisms to avoid those high drift points on old data patterns. It’s not just about training; it’s about the structure of the knowledge store itself.

Paper summary: Jane: So, moving toward the conclusion of this paper, what's the main point they are trying to drive home with this comparison between N-gram and LoRA? What's really new here?

Tom: The paper is titled "Representation Drift in Continual Learning: LoRA versus N-gram Memory." The authors are comparing these three update mechanisms under identical data and budgets to determine where new knowledge should be written for a language model that keeps learning online (<ref:2610.14921#pg1>).

Lu: The main implication is that N-gram memory leaves the base model nearly untouched, and its big-K drift is twenty-six percent/fifteen percent on math and code inputs against sixty-four percent/thirty-five percent for merged LoRA and ninety-seven percent/thirty-nine percent for shared LoRA (<ref:2610.14921#pg1>).

Meng: It seems the authors are arguing that N-gram memory is actually a more stable way to absorb new knowledge into an existing model compared to the other methods because its generic behavior stays essentially unchanged (<ref:2610.14921#pg1>).

Jane: So, thinking about what this means for us using these models every day, it suggests that if you use a native N-gram memory, you're less likely to erode the general capabilities of your model over time compared to the other ways of handling continual learning.

Tom: Exactly. The paper shows that for N-gram memory, generic drift stays small and interference concentrates on inputs that share rows with new tasks (<ref:2610.14921#pg8>). It’s a single source term picture where the injection change is small on generic inputs but large on old-task inputs that read rows later tasks rewrite (<ref:2610.14921#pg8>).

Lu: The authors are emphasizing that N-gram memory can have very small big-K drift because generic inputs rarely read rows the tasks write, and still have large small-ki drift because old-task inputs share rows with new tasks (<ref:2610.14921#pg8>).

Meng: From a production standpoint, that points toward a design choice for memory retrieval that favors stability for the core knowledge while accepting localized interference patterns. It's not about achieving perfect retention everywhere, but managing where the drift is happening.

Jane: So, to wrap up, this paper suggests that N-gram memory is a viable option because its big-K drift against general math and code inputs is relatively small compared to LoRA methods (<ref:2610.14921#pg1>).

Tom: And the conclusion they draw is that for online learning, updating only the N-gram memory changes the base model’s representations of general math and code inputs by twenty-six percent/fifteen percent against sixty-four percent/thirty-five percent for merged LoRA and ninety-seven percent/thirty-nine percent for shared LoRA (<ref:2610.14921#pg1>).

Lu: The title itself, "Representation Drift in Continual Learning: LoRA versus N-gram Memory," sets up the core tension we're seeing here about how these different systems handle knowledge acquisition over time.

Meng: It's a comparison of update strategies to see where new knowledge should be written for an AI that keeps learning online (<ref:2610.14921#pg1>).

Jane: So, this research gives us a clear picture of the trade-offs when we decide how to keep our models updated continuously. It shows that N-gram memory has specific behavior concerning drift and interference that is worth paying attention to (<ref:2610.14921#pg8>).

Conclusion: Tom: So we've been looking at this paper, "Representation Drift in Continual Learning: LoRA versus N-gram Memory." It’s comparing three ways to update a language model when it keeps learning online.

Jane: Right, so it looks at N-gram memory against training a shared LoRA adapter and training a new LoRA for every task. The main thing they're showing is where the new knowledge actually writes itself in the model.

Tom: Exactly, and they found that updating only the N-gram memory leaves the base model nearly untouched, especially for general math and code inputs. It also sees a much smaller big-K drift there compared to what happens with LoRA methods.

Jane: That means if you use that N-gram approach, your core understanding of things like math or coding isn't being eroded as quickly when the AI is constantly getting new tasks.

Tom: Precisely, and they show that while generic behavior stays stable, there’s this specific interference issue where old tasks share rows with new ones in the memory.

Jane: So, it’s not a total protection against forgetting; it’s more like localized friction between old and new data patterns. That's a key distinction to make.

Tom: Yeah, and Lalam confirms that this drift is tied to how much knowledge you lose when you add things on top. It’s a specific kind of retention loss driven by those overlapping inputs.

Jane: It makes me wonder how this applies to real-world systems where models are learning continuously from user interactions. What does that mean for the long-term stability of deployed AI?

Tom: Well, it suggests that if you want to keep the general capabilities solid while still allowing the model to adapt, you have to manage those specific row overlaps in your memory structure carefully. We need to figure out how much interference we can tolerate.