When Is 0.1% Enough? Analyzing the Combined Effects of Dimensionality Reduction and Quantization on Text Embedding Compression

arXiv:2606.01074 · cs.CL · Submitted 2026-05-31 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Hi, I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: We're looking at the paper: "When Is 0.1% Enough? Analyzing the Combined Effects of Dimensionality Reduction and Quantization on Text Embedding Compression".

Jane: Here's the gist of the paper. The gist: Combining dimensionality reduction and quantization enables substantially stronger compression than using either method alone, that in some settings embeddings can be reduced to as little as 0.1% of their original size with almost no performance degradation, and that the optimal compression strategy depends on the task.

Tom: The Problem and Motivation High-performing text embedding models often output high-dimensional real-valued vectors, resulting in substantial storage and computational costs.

Jane: This is especially problematic in retrieval and retrieval-augmented generation (RAG), where large collections of document embeddings must be stored and compared against query embeddings.

Tom: Two common approaches to embedding compression are quantization and dimensionality reduction.

Paper summary: Tom: Hey Jane, so we're looking at this paper today, "When Is zero point one percent Enough <ref:2606.01074#pg1,When Is 0.1% Enough>? Analyzing the Combined Effects of Dimensionality Reduction and Quantization on Text Embedding Compression." It seems like it’s diving into why combining these two compression techniques is better than using them separately.

Jane: Right, I wonder what they mean by that combination being better, Tom? The abstract says they systematically examine how dimensionality reduction and quantization jointly affect embedding compression across four MTEB task families and five pretrained models.

Lu: It’s interesting because the core idea is that while you can do either dimensionality reduction or quantization on its own, the interaction between them hasn't been well-studied yet. This paper investigates that gap by looking at four task families: classification, clustering, retrieval, and STS.

Meng: From a deployment standpoint, I’m always thinking about the practical trade-offs of storage versus performance degradation when we try to compress these vectors for things like RAG systems. So what did they actually find?

Tom: Well, the main finding is that combining dimensionality reduction and quantization enables substantially stronger compression than using either method alone. They found that in some settings, embeddings can be reduced to as little as zero point one percent of their original size with almost no performance degradation for classification tasks <ref:2606.01074#pg1,that in some settings, embeddings can be reduced to as little as>.

Jane: That sounds like a big win if it holds up across different models and tasks, Tom. So they tested this on four models and those four task families?

Tom: Exactly. They used models like gte-Qwen2-7B-instruct and E5-large-v2, testing them against classification, clustering, retrieval, and STS tasks. The experimental setup involved dimensionality reduction methods like Head truncation or PCA followed by ROR rotation, paired with bit-widths from one to thirty-two bits for quantization <ref:2606.01074#pg2>.

Lu: And the results show that the preferred balance between dimensionality reduction and bit-width actually changes depending on what you are compressing. For instance, retrieval seems to be more sensitive to compression than classification.

Paper summary: Meng: That makes sense because retrieval is all about preserving fine-grained similarity structure, right? If you lose too much of that detail, the search quality drops fast. So how does this translate to real-world deployment where we have different requirements for speed and accuracy?

Jane: The paper points out that for classification tasks, performance can be maintained even under substantial compression. They show that even when vectors are compressed to one/one hundred twenty-eight of their original size, the performance drop is limited, going from eighty-nine point five one down to eighty-nine point two six in one example <ref:2606.01074#pg2,vectors are compressed to 1/128 of their original size, the performance>.

Tom: And they go further with the classification task, showing that at a compression of one/one thousand twenty-four the performance stays at eighty-eight point six two and their analysis suggests that under certain conditions, performance degradation remains limited even when original embeddings are reduced to zero point one percent of their size.

Lu: That's a strong result because it means they aren't just trading storage for a huge hit in accuracy across the board; the combination works to mitigate that loss. But they also noted that the choice of dimensionality reduction method matters, with PCA followed by random orthogonal rotation being effective under more aggressive compression.

Meng: What about the other tasks? If retrieval is sensitive, what happens when you try to compress those vectors for a similarity search application? Does this combined approach still handle it well?

Jane: The paper indicates that retrieval was generally the most sensitive to embedding compression. They found that under the ninety-nine percent performance threshold, retrieval required the largest relative compression budget, which suggests it’s actually the least compressible task among those four <ref:2606.01074#pg2>.

Tom: And they also talked about how different formats for quantization can behave differently. The paper mentions that fixed low-bit floating-point formats can sometimes act unexpectedly for normalized high-dimensional embeddings, which highlights the importance of matching the quantizer to the embedding distribution.

Lu: So, looking at what this means for representation learning, it shows that there isn't one universal compression setting; you have to tailor it based on whether you are classifying or retrieving.

Meng: For production deployment, that’s crucial because we can't apply a single compression strategy everywhere. The paper suggests we need to think about which task family we are optimizing for when choosing our compression pipeline.

Paper summary: Jane: So, what’s the big picture here? What does this paper actually change for someone who isn't deep in the math? It seems like it confirms that combining these two techniques is a better starting point than just picking one or the other.

Tom: The implication is that we can achieve much better storage savings without sacrificing performance as much as we thought possible when using only one technique. They are showing how to get significant compression gains, potentially down to zero point one percent of the original size for classification, while keeping accuracy high on models like gte-Qwen2-7B-instruct.

Lu: The paper concludes that these two compression axes are complementary tools. It points out that classification and clustering often tolerate aggressive compression, but retrieval needs a larger retained subspace to keep the similarity structure intact.

Meng: For us in shipping models, this means we need a task-aware approach to compression budgeting. We can’t just use one setting for everything; we have to assess the required trade-off per application type.

Jane: It sounds like the authors are warning us that if you try to compress too much without considering both dimensions together, you might end up with a situation where performance drops significantly, especially for tasks like retrieval.

Tom: And they also pointed out some limitations in their study, saying they only evaluated a limited set of compression methods and didn't fully characterize the bit-width and dimensionality trade-off across every single task family.

Lu: They also mentioned that their calibration protocol was task-specific and transductive, meaning it might not perfectly capture real-world deployment scenarios for every possible embedding setup.

Meng: So, while this gives us a much stronger understanding of the combined effect, we still have to be careful because the study itself has its own limitations regarding the scope of compression methods tested.

Jane: I think that’s a fair assessment. The paper provides strong evidence for combining these techniques, showing it works better than expected for classification, but we still have to consider those caveats about how broad their testing was.

Conclusion: Tom: So we’re wrapping up this look at "When Is zero point one percent Enough? Analyzing the Combined Effects of Dimensionality Reduction and Quantization on Text Embedding Compression."

Jane: It really boils down to how you can save space and still keep the meaning intact when you compress those text embeddings.

Tom: The main takeaway is that combining dimensionality reduction with quantization gives you much stronger compression than using either method by itself.

Jane: They found that for classification tasks, you can actually reduce the storage footprint to as little as zero point one percent of the original size with almost no drop in performance for some models.

Tom: That's a big win because it shows that these two techniques aren't just competing; they complement each other in a way we hadn't fully explored before.

Jane: But they also stressed that the best strategy isn't one-size-fits-all; it really depends on the specific task you are trying to do.

Tom: Yeah, and retrieval, for instance, was shown to be much more sensitive to this compression than classification was. That means your compression budget needs to change depending on whether you're searching a database or just categorizing text.

Jane: It suggests that for some applications, like retrieval, you might need to keep a bigger part of the original embedding structure intact just to ensure the search quality doesn't tank.

Tom: The authors also pointed out that this isn't perfect either; they noted limitations with their compression methods and how their calibration setup was task-specific.

Jane: So what does this mean for us in production? It means we can get way more storage savings, but we still have to carefully choose the right compression settings based on whether the AI is classifying or searching.

Tom: Exactly. And understanding that trade-off between compression and performance is crucial for deploying these large models efficiently at scale.

Jane: That brings us to thinking about how we can actually predict which combination of dimensionality reduction and bit-width will work best for any given use case, which is what we’re going to look at next.

Riku Kisako Hayato Tsukagoshi, Ryohei Sasano

Graduate School of Informatics, Nagoya University

cs.CL

Submitted: 2026-05-31

Updated: 2026-10-20

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 75/100

The gist: The gist: Combining dimensionality reduction and quantization enables substantially stronger compression than using either method alone, that in some settings embeddings can be reduced to as little

Key concepts

Dimensionality Reduction: This technique aims to reduce the number of coordinates (dimensions) in a high-dimensional vector without losing too much important information. Methods tested included keeping only the first 'd' coordinates or using PCA followed by a random rotation, which simplifies the vector while retaining key structural features.
Quantization: Quantization reduces the numerical precision of each coordinate by using fewer bits to represent it. This lowers storage requirements but can introduce errors. The study tested various bit-width settings (1 to 32 bits) to find the best trade-off between compression ratio and performance loss.
MTEB Task Families: These are four distinct categories of text tasks used to test the compression methods: classification, clustering, retrieval, and STS. The paper found that different tasks have different sensitivities; for instance, retrieval requires more dimensions to maintain similarity structure compared to classification.
Combined Compression Effect: This refers to using both dimensionality reduction and quantization together instead of using either method alone. The researchers found this combination is superior, allowing embeddings to be compressed substantially—sometimes by 0.1% of the original size—while preserving performance better than single-method approaches.

Terminology

Quantization — 5× in this paper · explained in 15 episodes
Embeddings — 2× in this paper · explained in 2 episodes

Summary

The gist: Combining dimensionality reduction and quantization enables substantially stronger compression than using either method alone, that in some settings embeddings can be reduced to as little as 0.1% of their original size with almost no performance degradation, and that the optimal compression strategy depends on the task.

The Problem and Motivation

High-performing text embedding models often output high-dimensional real-valued vectors, resulting in substantial storage and computational costs. This is especially problematic in retrieval and retrieval-augmented generation (RAG), where large collections of document embeddings must be stored and compared against query embeddings. Two common approaches to embedding compression are quantization and dimensionality reduction. Quantization represents each coordinate with fewer bits, reducing numerical precision while keeping the dimensionality fixed. Dimensionality reduction instead reduces the number of retained coordinates. Despite progress in both directions, the interaction between quantization and dimensionality reduction remains insufficiently understood.

Experimental Setup

The study systematically examines the effectiveness of compressing text embeddings by combining dimensionality reduction and quantization, using four MTEB task families and four pretrained embedding models. The experiments were conducted on four MTEB task families: classification, clustering, retrieval, and STS. Four pretrained text embedding models were evaluated, including gte-Qwen2-7B-instruct and E5-large-v2. Dimensionality reduction methods compared included Head as the dimensionality reduction method, which keeps the first d coordinates, and PCA+ROR, which uses PCA-based dimensionality reduction followed by a deterministic random orthogonal rotation. Quantization settings covered six bit-width settings, with b bits per scalar in the set of 1, 2, 4, 8, 16, and 32.

Key Findings on Compression Effectiveness

The experimental results demonstrate that combining dimensionality reduction and quantization enables substantially stronger compression than using either method alone. In some settings embeddings can be reduced to as little as 0.1% of their original size with almost no performance degradation. For the classification task, even when the original vectors are compressed to 1/128 of their original size, the performance drop is limited. Even when they are compressed to 1/1024 of their original size, the performance remains at 88.62. Under certain conditions, we observed cases where performance degradation remained limited even when the original embeddings were reduced to 0.1% of their original size.

Task-Dependent Trade-offs

The effects of dimensionality reduction and quantization tend to vary depending on the task being processed. For classification tasks, performance can be maintained even under substantial compression, and Classification For classification tasks, performance can be maintained even under substantial compression. Conversely, retrieval was the most sensitive to embedding compression. Under the 99% performance threshold, retrieval required the largest relative compression budget, indicating that it was the least compressible. Overall, Overall, the dominant compression axis differs across task families.

Method Sensitivity Analysis

The choice of dimensionality reduction and quantization format matters. PCA followed by random orthogonal rotation is effective under aggressive compression, while Head-based truncation is more reliable when preserving almost the original performance. Furthermore, fixed low-bit floating-point formats can behave unexpectedly for normalized highdimensional embeddings, highlighting the importance of matching the quantizer to the embedding distribution. The analysis also showed that the instability of PCA-only compression is not specific to gte-Qwen2.

Conclusion

Across four embedding models and four MTEB task families, we found that these two compression axes are complementary. The best compression strategy, however, depends strongly on the task and model. Classification and clustering often tolerate aggressive compression, whereas retrieval generally requires a larger retained subspace to preserve fine-grained similarity structure. The paper concludes that the choice of dimensionality reduction and quantization format matters. Future work should develop methods for predicting suitable dimensionality and bit-width settings from small calibration sets, and should further analyze why different tasks and models exhibit different redundancy patterns.

Limitations

This study has three main limitations. First, we evaluate only a limited set of compression methods, focusing on Head-based truncation and PCA followed by random orthogonal rotation for dimensionality reduction, and simple post-training scalar quantization with a fixed set of bit-widths for quantization. Second, our evaluation is limited to a subset of MTEB, although we consider four major task families. Third, our calibration protocol is task-specific and transductive.

References

Hervé Abdi and Lynne J Williams. 2010. Principal component analysis

Dimitris Achlioptas. 2003. Database-friendly random projections: Johnson-Lindenstrauss with binary coins

Sanjeev Arora, Yingyu Liang, and Tengyu Ma. 2017. A Simple but Tough-to-Beat Baseline for Sentence Embeddings

Akari Asai, Timo Schick, Patrick Lewis, Xilun Chen, Gautier Izacard, Sebastian Riedel, Hannaneh Hajishirzi, and Wen-tau Yih. 2023. Task-aware Retrieval with Instructions

Laura Caspari et al. 2026. CoRECT: A Framework for Evaluating Embedding Compression Techniques at Scale

Daniel Cer et al. 2018. Universal Sentence Encoder

Alexis Conneau et al. 2017. Supervised Learning of Universal Sentence Representations from Natural Language Inference Data

Elias Frantar et al. 2022. GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers

Tianyu Gao et al. 2021. SimCSE: Simple Contrastive Learning of Sentence Embeddings

Naamán Huerga-Pérez et al. 2025. Optimization of embeddings storage for RAG systems using quantization and dimensionality reduction techniques

Taehee Jeong. 2024. 4bit-Quantization in VectorEmbedding for RAG

Aditya Kusupati et al. 2022. Matryoshka Representation Learning

Chankyu Lee et al. 2025a. NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Models

Jinhyuk Lee et al. 2025b. Gemini Embedding: Generalizable Embeddings from Gemini

Zehan Li et al. 2023. Towards General Text Embeddings with Multi-stage Contrastive Learning

Siyu Liao et al. 2020. Embedding Compression with Isotropic Iterative Quantization <ref:2606.

Improvements for AI systems

Improved storage efficiency for text embeddings by enabling substantially stronger compression than using either method alone, with some embeddings being reduced to as little as 0.1% of their original size with almost no performance degradation.
Enhanced retrieval performance by selecting an optimal combination of dimensionality reduction and quantization, as the paper shows that for retrieval, the task is the most sensitive to embedding compression and requires a larger retained subspace to preserve fine-grained similarity structure.
Task-adaptive compression strategy implementation: The system can dynamically choose between dimensionality reduction methods, noting that PCA followed by random orthogonal rotation is effective under aggressive compression, while using Head-based truncation when aiming to preserve almost the original performance.
Quantization format optimization: Instead of fixed low-precision formats, the system should utilize a distribution-adaptive lookup-table quantizer for 2-, 4-, and 8-bit settings, as this avoids excessive collapse of small but informative coordinates by matching representable levels to the empirical embedding distribution.
Model architecture flexibility: The system can leverage Matryoshka compatibility, as it shows that while not unique to specific models, it suggests that Matryoshka-style training may still improve the efficiency of truncation in certain cases, potentially leading to a smaller budget in specific settings like retrieval.

Abstract

Recent high-performing text embedding models often produce high-dimensional real-valued vectors, resulting in substantial storage and computational costs. To address this issue, embedding compression methods based on dimensionality reduction or quantization have been proposed; however, the effects of their combination have not been sufficiently investigated. In this paper, we systematically investigate how dimensionality reduction and quantization jointly affect embedding compression across four task families in the Massive Text Embedding Benchmark (MTEB), using five pretrained embedding models. Our results show that combining dimensionality reduction and quantization can preserve near-original performance with substantially smaller storage footprints than either method alone. In particular, for classification, several models retain at least 99% of their original classification score using less than 0.1% of the storage required by the original embeddings. The preferred balance between dimensionality and bit-width varies across tasks, with retrieval generally being more sensitive to compression.