D2: High-Capability Full-Duplex Spoken Language Models from Half-Duplex Backbones
cs.CL, cs.AI, eess.AS
Submitted: 2026-10-12
Updated: 2026-10-12
Comments: 17 pages, including references and appendices. Demo: https://d2-duplex-demo.github.io/
Project page: https://d2-duplex-demo.github.io
License: http://creativecommons.org/licenses/by/4.0/
The gist: Full-duplex spoken language models (FD-SLMs) support simultaneous listening and speaking, but often lag behind half-duplex spoken language models (HD-SLMs) in general capabilities.
Abstract
Full-duplex spoken language models (FD-SLMs) support simultaneous listening and speaking, but often lag behind half-duplex spoken language models (HD-SLMs) in general capabilities. We propose Dialogue-to-Duplex (D2), a framework that converts pretrained HD-SLMs into high-capability FD-SLMs at fine temporal granularity. D2 adapts the original audio encoder to streaming through decoder-free representation distillation and learns full-duplex interaction through supervised fine-tuning. We systematically study the effects of chunk size on full-duplex interaction, general capabilities, and inference computation. We apply D2 to two distinct HD-SLM architectures, Qwen3-Omni and LLaMA-Omni 2. At a 12.5Hz update rate, Qwen3-Omni-D2 achieves 99.2% turn-taking success with 521-ms latency, 99.5% interruption success, and a state-of-the-art VoiceBench score of 70.1.
Sources
- MiniCPM-o 4.5: Towards Real-Time Full-Duplex Omni-Modal Interaction
- Moshi: a speech-text foundation model for real-time dialogue
- Kimi-Audio Technical Report
- BayLing-Duplex: Native Full-Duplex Speech Dialogue with a Single Autoregressive LLM
- VITA: Towards Open-Source Interactive Omni Multimodal LLM
- Distilling the Knowledge in a Neural Network
- LoRA: Low-Rank Adaptation of Large Language Models
- Qwen3.5-Omni Technical Report
- AudioPaLM: A Large Language Model That Can Speak and Listen
- UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022
- Freeze-Omni: A Smart and Low Latency Speech-to-speech Dialogue Model with Frozen LLM
- Mini-Omni: Language Models Can Hear, Talk While Thinking in Streaming
- Qwen3-Omni Technical Report
- WeNet: Production oriented Streaming and Non-streaming End-to-End Speech Recognition Toolkit
- GLM-4-Voice: Towards Intelligent and Human-Like End-to-End Spoken Chatbot
- Instruction-Following Evaluation for Large Language Models
- Universal and Transferable Adversarial Attacks on Aligned Language Models
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering