Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Optimization of embeddings storage for RAG systems using quantization and dimensionality reduction techniques

Naamán Huerga-Pérez, Rubén Álvarez, Rubén Ferrero-Guillén, Alberto Martínez-Gutiérrez, Javier Díez-González

TL;DR

This work addresses the memory bottleneck of storing high-dimensional embeddings required by Retrieval-Augmented Generation. It systematically evaluates quantization formats (including float8 variants) and dimensionality reduction methods (PCA, Kernel PCA, UMAP, Random Projections, Autoencoders) on the MTEB benchmark. Key findings show that float8 quantization yields about $4\\times$ storage reduction with minimal performance loss and outperforms int8 at the same level, while standard PCA emerges as the most effective dimensionality reduction; combining moderate PCA with float8 enables roughly $8\\times$ total compression with less degradation than int8 alone. The authors also propose a visualization-based methodology to select Pareto-optimal configurations under a given memory budget, enabling scalable, memory-efficient deployment of RAG systems.

Abstract

Retrieval-Augmented Generation enhances language models by retrieving relevant information from external knowledge bases, relying on high-dimensional vector embeddings typically stored in float32 precision. However, storing these embeddings at scale presents significant memory challenges. To address this issue, we systematically investigate on MTEB benchmark two complementary optimization strategies: quantization, evaluating standard formats (float16, int8, binary) and low-bit floating-point types (float8), and dimensionality reduction, assessing methods like PCA, Kernel PCA, UMAP, Random Projections and Autoencoders. Our results show that float8 quantization achieves a 4x storage reduction with minimal performance degradation (<0.3%), significantly outperforming int8 quantization at the same compression level, being simpler to implement. PCA emerges as the most effective dimensionality reduction technique. Crucially, combining moderate PCA (e.g., retaining 50% dimensions) with float8 quantization offers an excellent trade-off, achieving 8x total compression with less performance impact than using int8 alone (which provides only 4x compression). To facilitate practical application, we propose a methodology based on visualizing the performance-storage trade-off space to identify the optimal configuration that maximizes performance within their specific memory constraints.

Optimization of embeddings storage for RAG systems using quantization and dimensionality reduction techniques

TL;DR

This work addresses the memory bottleneck of storing high-dimensional embeddings required by Retrieval-Augmented Generation. It systematically evaluates quantization formats (including float8 variants) and dimensionality reduction methods (PCA, Kernel PCA, UMAP, Random Projections, Autoencoders) on the MTEB benchmark. Key findings show that float8 quantization yields about storage reduction with minimal performance loss and outperforms int8 at the same level, while standard PCA emerges as the most effective dimensionality reduction; combining moderate PCA with float8 enables roughly total compression with less degradation than int8 alone. The authors also propose a visualization-based methodology to select Pareto-optimal configurations under a given memory budget, enabling scalable, memory-efficient deployment of RAG systems.

Abstract

Retrieval-Augmented Generation enhances language models by retrieving relevant information from external knowledge bases, relying on high-dimensional vector embeddings typically stored in float32 precision. However, storing these embeddings at scale presents significant memory challenges. To address this issue, we systematically investigate on MTEB benchmark two complementary optimization strategies: quantization, evaluating standard formats (float16, int8, binary) and low-bit floating-point types (float8), and dimensionality reduction, assessing methods like PCA, Kernel PCA, UMAP, Random Projections and Autoencoders. Our results show that float8 quantization achieves a 4x storage reduction with minimal performance degradation (<0.3%), significantly outperforming int8 quantization at the same compression level, being simpler to implement. PCA emerges as the most effective dimensionality reduction technique. Crucially, combining moderate PCA (e.g., retaining 50% dimensions) with float8 quantization offers an excellent trade-off, achieving 8x total compression with less performance impact than using int8 alone (which provides only 4x compression). To facilitate practical application, we propose a methodology based on visualizing the performance-storage trade-off space to identify the optimal configuration that maximizes performance within their specific memory constraints.
Paper Structure (2 sections, 3 figures)

This paper contains 2 sections, 3 figures.

Table of Contents

  1. Methodology for Optimal Configuration Selection
  2. Conclusions

Figures (3)

  • Figure 7: Performance loss vs. Compression Ratio for bge-small-en-v1.5 (left) and nomic-embed-text-v1.5 (right) across all tested quantization and dimensionality reduction combinations. The Y-axis shows the percentage performance drop relative to the float32 baseline without reduction. The X-axis shows the overall compression ratio achieved (log scale). The dashed line connects the configurations yielding the best observed performance for each compression level within each model.
  • Figure 8: Storage Size vs. QuoraRetrieval Performance for various configurations. Vertical lines indicate typical memory zones for different device classes. Allows selection of the best performing configuration within a given memory budget.
  • Figure 9: Storage Size vs. MTEB Weighted Average Performance for a hypothetical 125k embedding dataset. Illustrates how optimal configurations shift based on dataset size and available memory.