Exact Nearest-Neighbor Search on Energy-Efficient FPGA Devices
Patrizio Dazzi, William Guglielmo, Franco Maria Nardini, Raffaele Perego, Salvatore Trani
TL;DR
This work tackles exact $k$NN search in high-dimensional learned representations and proposes two FPGA-based architectures that share a single hardware configuration: FQ-SD for throughput by batching and streaming data, and FD-SQ for latency by streaming queries over an in-memory dataset. The designs rely on double buffering, a three-stage distance-computation pipeline, and a $k$-NN queue to achieve energy-efficient, exhaustive search with minimal accuracy loss. Experimental evaluation on three public datasets with vectors up to $d\approx 4{,}960$ demonstrates that the FPGA solutions outperform CPU baselines in throughput, latency, and energy, with scale-up factors up to $16.6\times$ latency improvement and $11.9\times$ energy savings, and can also beat CHIP-KNN and, in certain low-batch GPU scenarios, FAISS GPU. The results suggest practical impact for dense retrieval and other online applications demanding fast, exact $k$NN in resource-constrained or energy-sensitive environments.
Abstract
This paper investigates the usage of FPGA devices for energy-efficient exact kNN search in high-dimension latent spaces. This work intercepts a relevant trend that tries to support the increasing popularity of learned representations based on neural encoder models by making their large-scale adoption greener and more inclusive. The paper proposes two different energy-efficient solutions adopting the same FPGA low-level configuration. The first solution maximizes system throughput by processing the queries of a batch in parallel over a streamed dataset not fitting into the FPGA memory. The second minimizes latency by processing each kNN incoming query in parallel over an in-memory dataset. Reproducible experiments on publicly available image and text datasets show that our solution outperforms state-of-the-art CPU-based competitors regarding throughput, latency, and energy consumption. Specifically, experiments show that the proposed FPGA solutions achieve the best throughput in terms of queries per second and the best-observed latency with scale-up factors of up to 16.6X. Similar considerations can be made regarding energy efficiency, where results show that our solutions can achieve up to 11.9X energy saving w.r.t. strong CPU-based competitors.