Optimizing $k$ in $k$NN Graphs with Graph Learning Perspective

TL;DR

This work addresses the challenge of choosing a fixed $k$ in $k$-nearest neighbor graphs by proposing a variable-$k$ scheme guided by graph learning principles. It reframes per-node edge selection as a discrete, node-wise optimization using the Euclidean distance matrix $\mathbf{Z}$ and a per-node budget $\beta_i$, enabling automatic determination of the number of neighbors $k_i$ per node. The authors reveal a close relationship between their approach and sparse covariance estimation based graph learning, showing that the method acts as a fast, heuristic approximation to graph learning with substantially reduced computational cost. Experiments on real data, including point cloud denoising on ModelNet10, demonstrate that the proposed v$k$NNG yields sparser, well-connected graphs and improved denoising performance compared to fixed $k$NNG and NNK, while maintaining scalability to large graphs.

Abstract

In this paper, we propose a method, based on graph signal processing, to optimize the choice of $k$ in $k$-nearest neighbor graphs ($k$NNGs). $k$NN is one of the most popular approaches and is widely used in machine learning and signal processing. The parameter $k$ represents the number of neighbors that are connected to the target node; however, its appropriate selection is still a challenging problem. Therefore, most $k$NNGs use ad hoc selection methods for $k$. In the proposed method, we assume that a different $k$ can be chosen for each node. We formulate a discrete optimization problem to seek the best $k$ with a constraint on the sum of distances of the connected nodes. The optimal $k$ values are efficiently obtained without solving a complex optimization. Furthermore, we reveal that the proposed method is closely related to existing graph learning methods. In experiments on real datasets, we demonstrate that the $k$NNGs obtained with our method are sparse and can determine an appropriate variable number of edges per node. We validate the effectiveness of the proposed method for point cloud denoising, comparing our denoising performance with achievable graph construction methods that can be scaled to typical point cloud sizes (e.g., thousands of nodes).

Figures (3)

• Figure 1: Comparison of $k$NNGs.
• Figure 2: Geometric interpretation of the fixed and variable $k$NNGs in 2-D space. Note that $\beta_i$ in the v$k$NNG corresponds to the radius of the dotted bounding circle.
• Figure 3: Graphs constructed from the iris dataset. From left to right: $k_{\max}$NNG, graph learning, NNK, proposed v$k$NNG with $(k_{\min},k_{\max})=(3,7)$.