A Reproducibility Study of PLAID

TL;DR

PLAID addresses the efficiency-accuracy trade-off in late-interaction retrieval for ColBERTv2 by using centroid-based candidate generation and progressive pruning across three parameters: $nprobe$, $t_{cs}$, and $ndocs$. The paper reproduces PLAID's core results on standard benchmarks and reveals strong interdependencies among the parameters, showing that larger $ndocs$ generally yields better recall with modest latency increases. It additionally compares PLAID to a missing but important baseline: re-ranking a lexical BM25 system, and demonstrates that lexical-based pipelines (especially with LADR-style expansion) can offer superior latency-efficiency in low-latency regimes, though they may not fully match exhaustive ColBERTv2 search. A token-cluster analysis shows most PLAID clusters align with lexical matches, highlighting why lexical baselines are competitive and motivating potential hybrid approaches that combine PLAID-like semantic signals with fast lexical indexing for optimal performance.

Abstract

The PLAID (Performance-optimized Late Interaction Driver) algorithm for ColBERTv2 uses clustered term representations to retrieve and progressively prune documents for final (exact) document scoring. In this paper, we reproduce and fill in missing gaps from the original work. By studying the parameters PLAID introduces, we find that its Pareto frontier is formed of a careful balance among its three parameters; deviations beyond the suggested settings can substantially increase latency without necessarily improving its effectiveness. We then compare PLAID with an important baseline missing from the paper: re-ranking a lexical system. We find that applying ColBERTv2 as a re-ranker atop an initial pool of BM25 results provides better efficiency-effectiveness trade-offs in low-latency settings. However, re-ranking cannot reach peak effectiveness at higher latency settings due to limitations in recall of lexical matching and provides a poor approximation of an exhaustive ColBERTv2 search. We find that recently proposed modifications to re-ranking that pull in the neighbors of top-scoring documents overcome this limitation, providing a Pareto frontier across all operational points for ColBERTv2 when evaluated using a well-annotated dataset. Curious about why re-ranking methods are highly competitive with PLAID, we analyze the token representation clusters PLAID uses for retrieval and find that most clusters are predominantly aligned with a single token and vice versa. Given the competitive trade-offs that re-ranking baselines exhibit, this work highlights the importance of carefully selecting pertinent baselines when evaluating the efficiency of retrieval engines.

Figures (7)

• Figure 1: The Pareto frontier of PLAID for ColBERTv2 on TREC DL 2019 over the three parameters it introduces (nprobe, $t_{cs}$, and ndocs). Several operational points are labeled to highlight the interdependence of PLAID's parameters.
• Figure 2: The logical phases composing the candidate documents identification procedure in PLAID.
• Figure 3: Results of our study of PLAID's parameters nprobe, $t_{cs}$, and ndocs. Each row plots the same data points, with the colors representing each parameter value and the lines between them showing the effect with the other two parameters held constant. The dotted line shows the results of an exhaustive search, and the circled points highlight the three recommended settings from the original paper.
• Figure 4: Results of our baseline study. The lines connecting points for each approach represent its Pareto frontier.
• Figure 5: The distribution of Majority Token Proportions among clusters for ColBERTv2.