How the 2021 quantization algorithm quietly outperforms its 2026 successor

[3]is an online vector quantization method that gained wide public attention at ICLR 2026. To me, it looked very familiar. edenthe first quantization method introduced as a 1-bit method drive At NeurIPS 2021 [1] Generalized to any bit width in ICML 2022 [2]. Co-authored by me and Ran Ben-Basat, Yaniv Ben-Itzhak, Gal Mendelson, Michael Mitzenmacher, and Shay Vargaftik.

The TurboQuant paper presents two variations: TurboQuant-mse and TurboQuant products. In a detailed new comparison [5] TurboQuant-mse is a degenerate case of EDEN, and we show that EDEN variants consistently outperform their counterparts.

How EDEN quantizes vectors

Suppose you need to compress . $d$ -dimensional vector $×$ (gradient updates, embedding, KV cache entries) down to a few bits per coordinate. EDEN proceeds in four steps.

random rotation — Multiply by random orthogonal matrices. $\Pi$ . After rotation, the coordinates are distributed in the same way, and for large $d$ approximately Gaussian distribution.

scalar quantization — Round each rotated coordinate to one of the following: $2^b$ Lloyd-Max codebook levels trained with known Rotated coordinate distribution ( $b$ is the target number of bits per coordinate).

scale — multiply by scale factor $S$ .

Reverse rotation – apply $\Pi^\Top$ To recover the approximate value $\hat{x}$ of the original vector.

On the other hand, previous studies (e.g. Suresh et al. (2017)) [6]) Rotation was mainly used to reduce the coordinates. dynamic range (gap between maximum and minimum coordinate values), EDEN [1] To our knowledge, this was the first quantization scheme to take advantage of stronger facts about random rotations. That is, the rotated coordinates follow a known distribution. definitive A quantizer combined with a closed-form scale. Depending on the application, either minimize the MSE or unbias the estimate. Both scales are derived analytically, and this construction reduces MSE asymptotically compared to previous approaches.

Specifically, the two variations of EDEN differ only in the selection of: $S$ :

EDEN bias — set $S$ Set the reconstruction MSE to a closed-form value that minimizes it.

EDEN-Fairness — Select $S$ Therefore, the decompressed output will be correct on average ( $\mathbb{E}[\hat{x}] = x$ ), which is especially important when averaging many quantized vectors (distributed training, attention, etc.).

When aligned against EDEN, TurboQuant-mse matches on all steps except one: where EDEN derives the scale. $S$ Analytically, TurboQuant-mse targets MSE minimization but skips optimized scaling.

The following pseudocode shows the three side by side.

Figure 1: Pseudocode of EDEN biased, EDEN unbiased, and EDEN instantiated for TurboQuant-mse. The three are identical except for Step 5: Select S. Image by author [5].

Why optimal scale is valuable

The value of applying appropriate scale $S$ Increases with bit width. in $b = 1$ Bits, gaps are slight. in $d=128$ and $b = 4$ EDEN bias reduces MSE by 2.25% over TurboQuant-mse. These are the bit widths that practitioners actually use for embedding and KV caching.

Across dimensions 16 to 4096 and all bit widths tested $b \{1,2,3,4\}$ EDEN bias vNMSE (vector normalized MSE, $\mathbb{E}[\|x – \hat{x}\|^2] / \|x\|^2$ ) is lower than TurboQuant-mse in all cases (Figure 2). As the dimensions become very large, the optimal $S$ approaches 1 and the two algorithms converge, but a gap remains in the actual dimensions (128 to 1024).

Figure 2: Comparison of vNMSE and dimensionality comparing EDEN bias and TurboQuant-mse across bit widths. $b \{1,2,3,4\}$ (Panels from left to right). EDEN bias (optimizes scale factor) $S$ ) achieves lower error than TurboQuant-mse (modified). $S=1$ ) in all dimensions tested. It converges in high dimensions as an optimal curve. $S$ Approach 1. Image by author [5].

Unbiased compression: more than full bit savings

The above results are for the biased (MSE-minimizing) variant. Now consider the unbiased case, which is required by applications such as distributed training, approximate attention, and dot product search. $\mathbb{E}[\hat{x}] = x$ Because they average many quantized vectors.

EDEN-unbiased uses the same single-pass algorithm as EDEN-biased. $S$ selected for bias correction. TurboQuant-prod, an unbiased variant of TurboQuant, takes a different route. $(b-1)$ Reserve a bit for the biased TurboQuant-mse step and 1 bit for QJL (quantization Johnson-Lindenstrauss). [4] Correction of residuals (QJL is similar to EDEN) $b=1$ but the variance is larger).

EDEN-unbiased performs significantly better than TurboQuant-prod in all tested configurations. This gap traces three structural advantages of EDEN’s single-pass design.

EDEN optimizes scale. TurboQuant-prod inherits TurboQuant-mse $s=1$ Since it is the first stage, it is subject to the same MSE penalty.

EDEN’s 1-bit structure has lower variance than QJL. In large dimensions, EDEN’s 1-bit vNMSE converges to: $\pi/2 – 1 \approx. 0.57$ [1]while QJL converges as follows. $\pi/2 \approx. 1.57$ [4]approximately 2.75 times higher.

EDEN spends its entire bit budget on a single unbiased quantizer. TurboQuant-prod divides the budget into: $(b-1)$ Biased bit and one remaining bit. This empirically performs worse than spending everything. $b$ Number of bits in a single unbiased quantizer [5].

These effects occur in combination. result: 1-bit, 2-bit, and 3-bit EDEN-unbiased are more accurate than 2-bit, 3-bit, and 4-bit TurboQuant-prod, respectively (Figure 3). By replacing it with EDEN, you can drop a bit for each coordinate and still match the accuracy of TurboQuant-prod.

Figure 3: vNMSE and dimensionality comparison of EDEN-unbiased and TurboQuant-prod across bit widths. $b \{1,2,3,4\}$ (Panels from left to right). EDEN-unbiased achieves lower error in all dimensions. The gap is large enough that EDEN $b$ Bit often performs better than TurboQuant-prod. $b+1$ bit. Image by author [5].

About TurboQuant’s own benchmarks

The same situation applies to the standard ANN benchmarks evaluated by TurboQuant, Stanford’s GloVe pre-trained word vectors (Open Data Commons Public Domain Dedication and License v1.0) and the Qdrant benchmark. dbpedia-entities-openai3-text-embedding-3-large Embedding using TurboQuant’s publicly available evaluation code (Apache 2.0):

EDEN-biased achieves lower MSE than TurboQuant-mse, EDEN-unbiased achieves significantly lower dot product error than TurboQuant-prod, and the nearest-neighbor recall for both datasets favors EDEN (Figure 4).

Figure 4: Nearest neighbor recall at 2 and 4 bits per coordinate for GloVe and OpenAI3 embedding. EDEN-unbiased performs better than TurboQuant-prod in all four settings. Image by author [5].

Tip: Use EDEN. Optimal scaling is important

The EDEN scale connects the known rotated distribution to an analytically optimal quantizer. TurboQuant-mse retains EDEN rotation and codebook but is fixed $S=1$ This is why it is a strictly weak special case. TurboQuant-prod adds a 1-bit QJL stage on top of that. There, just by selecting a bias correction scale, EDEN-unbiased obtains the same characteristics with higher accuracy.

For compression targeting MSE (Model weight quantization, nearest neighbor search, KV cache): Calculate optimal scale with EDEN bias $S$ and consistently TurboQuant-mse ( teeth Eden with $S=1$ repaired).

For unbiased estimation (Variance mean estimation, approximate attention, dot product acquisition): EDEN unbiased significantly outperforms TurboQuant-prod’s bit-splitting strategy, boasting a margin of more than 1 bit per coordinate.

EDEN was originally developed for variance mean estimation in federated and distributed training. Subsequent work applied it, for example, to embedding compression for document re-ranking (SDR2022 [8]), adapted for NVFP4 LLM training (MS-EDEN in Quartet II2026 [10]), generalized to vector quantization for data-free LLM weight compression (higgs2025 [9]), which was then used for KV cache compression (Aqua KV2025 [11]).

EDEN implementations are available in PyTorch and TensorFlow, as well as Intel’s OpenFL. [7]and 1-bit variants of Google’s FedJax, TensorFlow Federated, and TensorFlow Model Optimization.

See notes for complete technical comparative analysis (all figures, detailed experimental methodology) using TurboQuant. [5].

Please refer to the original paper for the original derivation, proof, and further extensions. [1] [2].

References

S. Vargaftik, R. Ben-Basat, A. Portnoy, G. Mendelson, Y. Ben-Itzhak, M. Mitzenmacher, DRIVE: 1-bit variance mean estimation (2021), NeurIPS 2021.

S. Vargaftik, R. Ben-Basat, A. Portnoy, G. Mendelson, Y. Ben-Itzhak, M. Mitzenmacher, EDEN: Communication-Efficient and Robust Distributed Mean Estimation for Federated Learning (2022), ICML 2022.

A. Zandieh, M. Daliri, A. Hadian, V. Mirrokni, TurboQuant: Online Vector Quantization with Near-Optimal Distortion Rates (2026), ICLR 2026.

A. Zandieh, M. Daliri, I. Han, QJL: 1-bit quantized JL transform for zero-overhead KV cache quantization (2024), arXiv:2406.03482.

R. Ben-Basat, Y. Ben-Itzhak, G. Mendelson, M. Mitzenmacher, A. Portnoy, S. Vargaftik, Notes on TurboQuant and the early DRIVE/EDEN work line (2026), arXiv:2604.18555.

AT Suresh, FX Yu, S. Kumar, HB McMahan, Variance-Mean Estimation with Limited Communication (2017), ICML 2017.

VMware Open Source Blog, VMware Research Group’s EDEN becomes part of OpenFL (November 2022).

N. Cohen, A. Portnoy, B. Fetahu, A. Ingber, SDR: Efficient Neural Re-ranking using Succinct Document Representation (2022), ACL 2022.

V. Malinovskii, A. Panferov, I. Ilin, H. Guo, P. Richtárik, D. Alistarh, HIGGS: Pushing the Limits of Large Language Model Quantization via the Linearity Theorem (2025), NAACL 2025.

A. Panferov, E. Schultheis, S. Tabesh, D. Alistarh, Quartet II: Accurate LLM pretraining in NVFP4 with improved unbiased gradient estimation (2026), arXiv:2601.22813.

A. Shutova, V. Malinovskii, V. Egiazarian, D. Kuznedelev, D. Mazur, N. Surkov, I. Ermakov, D. Alistarh, Cache Me If You Must: Adaptive Key-Value Quantization for Large Language Models (2025), ICML 2025.