Safe at One Loop, Risky at Another: Aligning Safety Across Recurrent Depths in Looped Language Models

summary

The gist

The gist The safety of a LoopLM cannot be inferred from a single recurrent depth, motivating safety alignment across recurrent computation.

In short

Looped Language Models (LoopLMs) expose outputs at multiple inference depths, creating safety challenges: safety varies across depths and jailbreaks transfer between them. SafeBridge is a preference optimization method that aligns these cross-depth safety issues by combining depth-specific modulation with selective state bridging, significantly reducing attack success rates while maintaining model utility.

Key concepts

Looped Language Models (LoopLMs): These are language models where the output is exposed at multiple different inference depths due to their recurrent structure. This means a single query can yield different results depending on how deep the model processes it, which complicates safety evaluation.
Safety Variation Across Inference Depths: This challenge shows that an attack's success rate changes depending on which inference depth is targeted. Attacks might succeed against one depth but fail against another, making safety unpredictable across the model's recurrent steps.
CrossDepth Jailbreak Transfer: This occurs when a jailbreak crafted to bypass safety at one inference depth can also successfully bypass safety at other depths. This means attackers have multiple ways to exploit vulnerabilities within the model's various recurrent layers.

Terminology used across episodes

Reinforcement Learning (RL) — explained in 25 episodes
Sample Complexity — explained in 20 episodes
Retrieval-Augmented Generation (RAG) — explained in 19 episodes
Federated Learning — explained in 17 episodes
Flow Matching — explained in 17 episodes
Quantum Fisher Information (QFI) — explained in 17 episodes

This episode discusses

The paper

Safe at One Loop, Risky at Another: Aligning Safety Across Recurrent Depths in Looped Language Models · Read on arXiv

ShanghaiTech University · Microsoft Research Asia

Transcript

Introduction to the show: ident: Security Radio. Generated commentary on the latest security and cryptography papers.

Nadia: Today's paper: "Safe at One Loop, Risky at Another".

Elias: The gist The safety of a LoopLM cannot be inferred from a single recurrent depth, motivating safety alignment across recurrent computation.

Nadia: First, who's behind it and why it matters.

Paper summary: Nadia: We're moving into this next part where we look at what's actually happening in this paper, 'Safe at One Loop, Risky at Another: Aligning Safety Across Recurrent Depths in Looped Language Models'. This paper is fundamentally concerned with the safety of Looped Language Models which scale up using shared parameters.

Elias: The central thesis here is that since a single LoopLM has outputs across multiple inference depths, we must investigate whether safety is maintained throughout all those recurrent steps, because prior tests just didn't show us this robustness yet.

Priya: They point out the two key issues they found: first, safety can vary depending on the inference depth you check for a specific jailbreak input.

Nadia: And second, attacks built against one source depth can jump over and work on other depths, creating multiple attack sources for any target depth you pick.

Elias: It matters because this means current safety checks aren't enough; we need a way to align safety across these different recurrent depths instead of just hoping the final output is safe.

Priya: So they are setting out to study this variation and alignment across recurrent depths, which is something that hasn't been much explored in the field yet.

Conclusion: Nadia: Looking at the title, 'Safe at One Loop, Risky at Another', it really captures the tension they found: safety isn't guaranteed just because you check one level of computation; it depends entirely on which depth you are looking at.

Elias: And the authors, Yi Wang and others, they are essentially pointing out that scaling up with Looped Language Models introduces a new complexity where we can't treat every recurrent step as having the same safety guarantee.

Priya: What this means for me is that future work needs to focus on building systems that inherently understand this multi-depth nature of the computation, not just trying to patch it after training.

Nadia: This work motivates us to think about how we align safety across recurrent computation rather than just focusing on a single output depth for these powerful models.

More episodes

← Home