Learning Filter-Aware Distance Metrics for Nearest Neighbor Search with Multiple Filters

TL;DR

This work tackles the Filtered-ANN problem with AND-style constraints by learning a data-driven distance that blends vector similarity with filter satisfaction. It formulates a linear program to determine a nonnegative weight $w_m$ in the distance $D(q,v) = d(q,v) + w_m \cdot (1 - m(q,v))$, where $m(q,v)$ measures filter alignment, and integrates this learned metric into a graph-based index (FilteredDiskANN) via WeightedGreedySearch. Ground-truth preferences are mined through exhaustive scans to train the LP, and the solution guides both index construction and query-time search, with a planning strategy to switch to exact search for highly selective queries. Empirical results on YFCC1M and Wikipedia show consistent 5–10% recall gains over fixed-penalty baselines, while reducing candidate evaluations and preserving unfiltered search quality, highlighting improved generalization and practical impact for filtered ANN workloads.

Abstract

Filtered Approximate Nearest Neighbor (ANN) search retrieves the closest vectors for a query vector from a dataset. It enforces that a specified set of discrete labels $S$ for the query must be included in the labels of each retrieved vector. Existing graph-based methods typically incorporate filter awareness by assigning fixed penalties or prioritizing nodes based on filter satisfaction. However, since these methods use fixed, data in- dependent penalties, they often fail to generalize across datasets with diverse label and vector distributions. In this work, we propose a principled alternative that learns the optimal trade-off between vector distance and filter match directly from the data, rather than relying on fixed penalties. We formulate this as a constrained linear optimization problem, deriving weights that better reflect the underlying filter distribution and more effectively address the filtered ANN search problem. These learned weights guide both the search process and index construction, leading to graph structures that more effectively capture the underlying filter distribution and filter semantics. Our experiments demonstrate that adapting the distance function to the data significantly im- proves accuracy by 5-10% over fixed-penalty methods, providing a more flexible and generalizable framework for the filtered ANN search problem.

Figures (4)

• Figure 1: Recall vs. Average Distance Comparison on the Wikipedia-35M dataset. The learned weight $w_m = 0.204148$, and 464 queries proceeded to the graph search.
• Figure 2: Recall vs. Average Distance Comparison on the YFCC dataset. The learned weight is $w_m = 0.017787$, and 1,727 queries proceeded to the graph search.
• Figure 3: Unfiltered Search on different builds, this shows our index construction method preserves the quality of unfiltered search.
• Figure 4: Percentile refers to the ranking of queries based on their expected selectivity, with percentile 0 being the most selective (i.e., queries that pass for very few items) and percentile 99 being the least selective (i.e., queries that pass for the most items). Pass rate is defined as the total number of data points that satisfy the query conditions.