Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

kNN-Graph: An adaptive graph model for $k$-nearest neighbors

Jiaye Li, Gang Chen, Hang Xu, Shichao Zhang

TL;DR

The paper tackles the scalability challenge of kNN in large-scale applications by decoupling inference latency from the dataset size. It introduces kNN-Graph, combining kernelized self-representation for adaptive neighborhood learning with a hierarchical HNSW index that stores precomputed voting results, enabling logarithmic-time inference. The approach yields real-time classification without sacrificing accuracy, demonstrated across six diverse datasets against eight baselines, and achieves superior macro-metrics and dramatic speedups, especially on high-dimensional data. The work provides a practical structural paradigm for graph-based nonparametric learning, shifting heavy computation to training and delivering efficient, scalable inference for real-time tasks.

Abstract

The k-nearest neighbors (kNN) algorithm is a cornerstone of non-parametric classification in artificial intelligence, yet its deployment in large-scale applications is persistently constrained by the computational trade-off between inference speed and accuracy. Existing approximate nearest neighbor solutions accelerate retrieval but often degrade classification precision and lack adaptability in selecting the optimal neighborhood size (k). Here, we present an adaptive graph model that decouples inference latency from computational complexity. By integrating a Hierarchical Navigable Small World (HNSW) graph with a pre-computed voting mechanism, our framework completely transfers the computational burden of neighbor selection and weighting to the training phase. Within this topological structure, higher graph layers enable rapid navigation, while lower layers encode precise, node-specific decision boundaries with adaptive neighbor counts. Benchmarking against eight state-of-the-art baselines across six diverse datasets, we demonstrate that this architecture significantly accelerates inference speeds, achieving real-time performance, without compromising classification accuracy. These findings offer a scalable, robust solution to the long-standing inference bottleneck of kNN, establishing a new structural paradigm for graph-based nonparametric learning.

kNN-Graph: An adaptive graph model for $k$-nearest neighbors

TL;DR

The paper tackles the scalability challenge of kNN in large-scale applications by decoupling inference latency from the dataset size. It introduces kNN-Graph, combining kernelized self-representation for adaptive neighborhood learning with a hierarchical HNSW index that stores precomputed voting results, enabling logarithmic-time inference. The approach yields real-time classification without sacrificing accuracy, demonstrated across six diverse datasets against eight baselines, and achieves superior macro-metrics and dramatic speedups, especially on high-dimensional data. The work provides a practical structural paradigm for graph-based nonparametric learning, shifting heavy computation to training and delivering efficient, scalable inference for real-time tasks.

Abstract

The k-nearest neighbors (kNN) algorithm is a cornerstone of non-parametric classification in artificial intelligence, yet its deployment in large-scale applications is persistently constrained by the computational trade-off between inference speed and accuracy. Existing approximate nearest neighbor solutions accelerate retrieval but often degrade classification precision and lack adaptability in selecting the optimal neighborhood size (k). Here, we present an adaptive graph model that decouples inference latency from computational complexity. By integrating a Hierarchical Navigable Small World (HNSW) graph with a pre-computed voting mechanism, our framework completely transfers the computational burden of neighbor selection and weighting to the training phase. Within this topological structure, higher graph layers enable rapid navigation, while lower layers encode precise, node-specific decision boundaries with adaptive neighbor counts. Benchmarking against eight state-of-the-art baselines across six diverse datasets, we demonstrate that this architecture significantly accelerates inference speeds, achieving real-time performance, without compromising classification accuracy. These findings offer a scalable, robust solution to the long-standing inference bottleneck of kNN, establishing a new structural paradigm for graph-based nonparametric learning.
Paper Structure (38 sections, 1 theorem, 7 equations, 6 figures, 6 tables, 2 algorithms)

This paper contains 38 sections, 1 theorem, 7 equations, 6 figures, 6 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. $k$ Selection
  4. Efficient Nearest Neighbor Search and Indexing
  5. Integrated Search and $k$ Selection Paradigms
  6. Methods
  7. Notations and Problem Formulation
  8. Adaptive Neighborhood Learning via Kernelized Self-Representation
  9. The Kernelized Optimization Objective
  10. Density-Aware Adaptive Regularization
  11. Hierarchical Graph Indexing with Precomputed Decisions
  12. Embedding Precomputed Intelligence into HNSW
  13. Logarithmic-Time Inference Mechanism
  14. Algorithmic Implementation and Workflow
  15. Theoretical Guarantees
  16. ...and 23 more sections

Key Result

Proposition 1

Let $\mathbf K$ be the kernel matrix and $\mathbf k_j$ be its $j$-th column. If the minimum regularization parameter satisfies $\lambda_{\min} < \min_j \|\mathbf K(\cdot, j)\|_2^2$, then every sample $\mathbf x_j$ is guaranteed to have at least one neighbor ($w_{ij} \neq 0$ for some $i$), preventing

Figures (6)

  • Figure 1: Schematic illustration of the Adaptive Graph Model for $k$NN.a Traditional $k$NN relies on a fixed $k$ and performs computationally expensive search and voting at runtime. b During the training phase, the proposed method adaptively learns the optimal neighborhood structure for each sample (adjusting $k$ to local density) and embeds a precomputed consensus label directly into each node. c During inference, a query utilizes a Hierarchical Navigable Small World (HNSW) structure to rapidly navigate to the nearest precomputed node and instantly retrieve its stored label, achieving logarithmic-time classification without real-time voting.
  • Figure 2: Schematic overview of the proposed $k$NN-Graph framework. The method comprises three main stages: kernelized self-expression learning, HNSW graph construction, and inference. In the kernelized self-expression learning stage, the original data are mapped into a high-dimensional kernel space using a kernel function to capture nonlinear relationships among data points. This allows the self-expression model to determine the optimal $k$ value and the $k$ nearest neighbors for each training sample. The HNSW graph constructs an efficient indexing structure to rapidly retrieve the nearest neighbors of test samples, including the optimal $k$ value, neighbor indices, and weight-based voting labels. During the inference stage, the nearest neighbors of test samples are directly obtained from the HNSW graph, and the precomputed class labels are used as predictions.
  • Figure 3: Average Macro-Recall of all algorithms over ten runs on six public datasets.
  • Figure 4: Average Macro-F1 Score of all algorithms over ten runs on six public datasets.
  • Figure 5: Ablation analysis demonstrating the impact of Adaptive Neighborhood Learning. Pair-wise performance comparison between the baseline HNSW (w/o Adaptive) and the proposed $k$NN-Graph (w/ Adaptive) across six benchmark datasets. The slope charts illustrate the performance evolution in terms of a Classification Accuracy, b Macro Precision, c Macro Recall, and d Macro F1-Score. The connecting lines trace the improvement for each dataset, with the specific positive gain ($+\Delta$) annotated in the center. The strictly upward trajectories across all metrics highlight the effectiveness of jointly learning the optimal neighbor count and weights, confirming that the adaptive mechanism significantly enhances discriminative power compared to the static graph structure of HNSW.
  • ...and 1 more figures

Theorems & Definitions (2)

  • Proposition 1: Connectivity Stability
  • proof