Reconstruct! Don't Encode: Self-Supervised Representation Reconstruction Loss for High-Intelligibility and Low-Latency Streaming Neural Audio Codec
eess.AS, cs.AI
Submitted: 2026-03-06
Updated: 2026-09-01
Comments: Submitted to TASLP
Code: https://github.com/jhcodec843/jhcodec
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- Moshi: a speech-text foundation model for real-time dialogue
- High Fidelity Neural Audio Compression
- WavTokenizer: an Efficient Acoustic Discrete Codec Tokenizer for Audio Language Modeling
- MagiCodec: Simple Masked Gaussian-Injected Codec for High-Fidelity Reconstruction and Generation
- Scaling Transformers for Low-Bitrate High-Quality Speech Coding
- FocalCodec-Stream: Streaming Low-Bitrate Speech Coding via Causal Distillation
- SpeechTokenizer: Unified Speech Tokenizer for Speech Large Language Models
- SeamlessM4T: Massively Multilingual & Multimodal Machine Translation
- BigCodec: Pushing the Limits of Low-Bitrate Neural Speech Codec
- XY-Tokenizer: Mitigating the Semantic-Acoustic Conflict in Low-Bitrate Speech Codecs
- TS3-Codec: Transformer-Based Simple Streaming Single Codec
- High-Fidelity Simultaneous Speech-To-Speech Translation
- When Worse is Better: Navigating the compression-generation tradeoff in visual tokenization
- Language Model Beats Diffusion -- Tokenizer is Key to Visual Generation
- Heptapod: Language Modeling on Visual Signals
- GLU Variants Improve Transformer
- Layer Normalization
- Effective Context in Neural Speech Models
- Llasa: Scaling Train-Time and Inference-Time Compute for Llama-based Speech Synthesis
- Vector-quantized Image Modeling with Improved VQGAN
Related papers
- Rethinking Entropy Allocation in LLM-based ASR: Understanding the Dynamics between Speech Encoders and LLMs
- X-VC: Zero-shot Streaming Voice Conversion in Codec Space
- Autoregressive Guidance of Deep Spatially Selective Filters using Bayesian Tracking for Efficient Extraction of Moving Speakers
- LiveAnon: Improving Privacy and Utility in Streaming Speaker Anonymization with Codec Language Models
- EarLog: Opportunistic Ear Physiology Sensing toward Personalized Hearing Risk Awareness
- VM-ARRAYDPS: Virtual Microphone Augmented Diffusion Posterior Sampling for Unsupervised Blind Speech Separation