Optimizing Singular Spectrum for Large Language Model Compression
cs.CL
Submitted: 2025-02-20
Updated: 2026-10-16
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
- Ternary Singular Value Decomposition as a Better Parameterized Form in Linear Mapping
- Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge
- BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
- Hessian Eigenvectors and Principal Component Analysis of Neural Network Weight Matrices
- Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding
- Mistral 7B
- Scaling Laws for Neural Language Models
- LLaMA: Open and Efficient Foundation Language Models
- Llama 2: Open Foundation and Fine-Tuned Chat Models
- Efficient Large Language Models: A Survey
- ASVD: Activation-aware Singular Value Decomposition for Compressing Large Language Models
- OPT: Open Pre-trained Transformer Language Models
- BlockPruner: Fine-grained Pruning for Large Language Models
- A Survey on Model Compression for Large Language Models
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering