Understanding and Mitigating Token-Pruning-Induced Vulnerabilities in VLMs
cs.CR, cs.LG
Submitted: 2026-10-07
Updated: 2026-10-07
Comments: Accepted at ICML 2026. 16 pages
Journal ref: Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:128879-128894, 2026
License: http://creativecommons.org/licenses/by/4.0/
The gist: Token-Pruning accelerates Vision-Language Models by removing redundant visual tokens, yet its safety implications remain underexplored.
Terminology
Abstract
Token-Pruning accelerates Vision-Language Models by removing redundant visual tokens, yet its safety implications remain underexplored. In this work, we present the first comprehensive safety evaluation of Token-Pruning mechanisms and find that: most pruning strategies significantly degrade safety as pruning ratios increase, whereas Query-based Compression shows the opposite, with extreme pruning (up to 99.8%), unexpectedly improves model safety. This sharp contrast prompts a key question: How do different Token-Pruning strategies reshape model safety behavior, and is it possible to enhance safety without sacrificing acceleration? To answer this, we identify an unrecognized mechanism, termed Pruning-Induced Malicious Amplification, where removal of background tokens triggers a side effect: forcing the model's attention to collapse onto a few retained malicious anchors within the foreground, inadvertently amplifying their toxic semantics under jailbreak. To address that, we propose an inference-time and plug-and-play Safety-Aware Pruning (SAP) mechanism that counteracts such dominance via three steps: (1) identifying malicious anchors, (2) restoring pruned benign tokens, and (3) reallocating excessive attention from malicious anchors to benign tokens. Extensive experiments across three safety and four utility benchmarks demonstrate that SAP mitigates pruning-induced vulnerabilities, i.e., reducing ASR by up to 62%, without compromising efficiency or utility.
Sources
- Model Compression vs. Adversarial Robustness: An Empirical Study on Language Models for Code
- Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond
- Matryoshka Query Transformer for Large Vision-Language Models
- Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations
- Token Reduction Should Go Beyond Efficiency in Generative Models -- From Vision, Language to Multimodality
- Visual Instruction Tuning
- HiPrune: Hierarchical Attention for Efficient Token Pruning in Vision-Language Models
- MM-SafetyBench: A Benchmark for Safety Evaluation of Multimodal Large Language Models
- DeepSeek-VL: Towards Real-World Vision-Language Understanding
- JailBreakV: A Benchmark for Assessing the Robustness of MultiModal Large Language Models against Jailbreak Attacks
- Less is More: A Simple yet Effective Token Reduction Method for Efficient Multi-modal LLMs
- Beyond Text-Visual Attention: Exploiting Visual Cues for Effective Token Pruning in VLMs
- MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models
Related papers
- SoK: AI-Augmented Binary Reversing
- Relaxed Sender Anonymity for CBDC Interbank Settlement: A Zero-Knowledge Approach on Permissioned EVM
- Calibration-Family Overfit: Why Trusted Sabotage Monitors Don't Transfer Across Lineages
- Estimating the Social Cost of Corporate Data Breaches
- Efficient Fuzzy PSI under One-Sided Assumptions
- Sealing the Audit-Runtime Gap for LLM Skills