Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

RAE: A Neural Network Dimensionality Reduction Method for Nearest Neighbors Preservation in Vector Search

Han Zhang, Dongfang Zhao

TL;DR

The paper tackles the challenge of preserving nearest-neighbor structure when compressing high-dimensional embeddings for fast vector search. It introduces Regularized Auto-Encoder (RAE), a simple two-layer linear auto-encoder with a Frobenius-norm regularizer that controls the singular value spectrum; the resulting bound on norm distortion is derived via the Rayleigh quotient, with the regularization coefficient $λ$ tuning neighborhood preservation. Theoretical results connect k-NN preservation to norm-bounded transformations and show that the condition number $\kappa(W)$ governs distortion, enabling provable guarantees. Empirical results across diverse datasets demonstrate that RAE achieves superior k-NN preservation with competitive inference speed, making it a practical drop-in DR method for large-scale vector retrieval and RAG workflows.

Abstract

While high-dimensional embedding vectors are being increasingly employed in various tasks like Retrieval-Augmented Generation and Recommendation Systems, popular dimensionality reduction (DR) methods such as PCA and UMAP have rarely been adopted for accelerating the retrieval process due to their inability of preserving the nearest neighbor (NN) relationship among vectors. Empowered by neural networks' optimization capability and the bounding effect of Rayleigh quotient, we propose a Regularized Auto-Encoder (RAE) for k-NN preserving dimensionality reduction. RAE constrains the network parameter variation through regularization terms, adjusting singular values to control embedding magnitude changes during reduction, thus preserving k-NN relationships. We provide a rigorous mathematical analysis demonstrating that regularization establishes an upper bound on the norm distortion rate of transformed vectors, thereby offering provable guarantees for k-NN preservation. With modest training overhead, RAE achieves superior k-NN recall compared to existing DR approaches while maintaining fast retrieval efficiency.

RAE: A Neural Network Dimensionality Reduction Method for Nearest Neighbors Preservation in Vector Search

TL;DR

The paper tackles the challenge of preserving nearest-neighbor structure when compressing high-dimensional embeddings for fast vector search. It introduces Regularized Auto-Encoder (RAE), a simple two-layer linear auto-encoder with a Frobenius-norm regularizer that controls the singular value spectrum; the resulting bound on norm distortion is derived via the Rayleigh quotient, with the regularization coefficient tuning neighborhood preservation. Theoretical results connect k-NN preservation to norm-bounded transformations and show that the condition number governs distortion, enabling provable guarantees. Empirical results across diverse datasets demonstrate that RAE achieves superior k-NN preservation with competitive inference speed, making it a practical drop-in DR method for large-scale vector retrieval and RAG workflows.

Abstract

While high-dimensional embedding vectors are being increasingly employed in various tasks like Retrieval-Augmented Generation and Recommendation Systems, popular dimensionality reduction (DR) methods such as PCA and UMAP have rarely been adopted for accelerating the retrieval process due to their inability of preserving the nearest neighbor (NN) relationship among vectors. Empowered by neural networks' optimization capability and the bounding effect of Rayleigh quotient, we propose a Regularized Auto-Encoder (RAE) for k-NN preserving dimensionality reduction. RAE constrains the network parameter variation through regularization terms, adjusting singular values to control embedding magnitude changes during reduction, thus preserving k-NN relationships. We provide a rigorous mathematical analysis demonstrating that regularization establishes an upper bound on the norm distortion rate of transformed vectors, thereby offering provable guarantees for k-NN preservation. With modest training overhead, RAE achieves superior k-NN recall compared to existing DR approaches while maintaining fast retrieval efficiency.

Paper Structure

This paper contains 22 sections, 24 equations, 1 figure, 2 tables.

Table of Contents

  1. Introduction
  2. Background and Motivation
  3. Proposed Work
  4. Contributions
  5. Related Work
  6. Classical Dimensionality Reduction Methods
  7. Manifold Learning and Nonlinear Methods
  8. Deep Learning Methods for Dimensionality Reduction
  9. Methodology
  10. Problem Formulation
  11. RAE Architecture
  12. Theoretical Foundation
  13. From k-NN Preservation to Norm-Bounded Transformation
  14. Rayleigh Quotient Properties for Singular Value-Related Upper Bound
  15. Experiments
  16. ...and 7 more sections

Figures (1)

  • Figure 1: Weight decay analysis on IMDb (Euclidean, 256d) and Flickr30k (cosine, 768d) datasets. Each row shows k-NN accuracy for different k values, singular value spectrum ($\sigma_{max}$ and $\sigma_{min}$), and condition number $\kappa(W)$ as functions of weight decay $\lambda$. The consistent pattern across datasets—optimal accuracy at minimal condition number—demonstrates the robustness of our regularization strategy.

Theorems & Definitions (2)

  • Definition 1: k-NN Preservation Task
  • Definition 2: k-NN Preservation Accuracy