Label Collapse in Diffusion Language Model In-Context Learning
cs.CL
Submitted: 2026-10-12
Updated: 2026-10-12
Comments: Accepted by NeurIPS 2026 DiffuLM workshop
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- Language Models are Few-Shot Learners
- Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity
- Calibrate Before Use: Improving Few-Shot Performance of Language Models
- Large Language Diffusion Models
- Masks Can Be Distracting: On Context Comprehension in Diffusion Language Models
- Surface Form Competition: Why the Highest Probability Answer Isn't Always Right
- Improved Large Language Diffusion Models
- The Llama 3 Herd of Models
- Beyond AdS$_2$/dCFT$_1$: Insertions in Two Wilson Loops
- Many-Shot In-Context Learning
- On the Loss of Context-awareness in General Instruction Fine-tuning
- How Robust are LLMs to In-Context Majority Label Bias?
- Mitigating Label Biases for In-context Learning
- In-context Learning and Induction Heads
- Understanding Task Vectors in In-Context Learning: Emergence, Functionality, and Limitations
- The Geometry of Prompting: Unveiling Distinct Mechanisms of Task Adaptation in Language Models
- Simple and Effective Masked Diffusion Language Models
- Dream 7B: Diffusion Large Language Models
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering