Top-P Masking for Cross Language Information Retrieval

TL;DR

The paper tackles cross-language information retrieval (CLIR) by promoting sparse, embedding-space representations to enable scalable indexing. It introduces Top-P Dynamic Masking, a nucleus-sampling–inspired post-processing scheme that selects a dynamic set $S_p$ of vocabulary terms defined by $\sum_{j \in S_p} w_j \ge p \sum_{j \in V} w_j$. Empirical results on NeuCLIR indicate that Top-P Dynamic Masking yields competitive or superior $mAP$ compared with Top-K masking while offering favorable throughput, highlighting the value of adaptive sparsity for CLIR. The findings clarify how term-selection distributions differ between documents and queries and point to practical gains for deploying sparse embeddings in multilingual search systems.

Abstract

Top-K masking schemes have been proposed as a method to promote sparse representations in Information Retrieval (IR) tasks, as a simple alternative to Floating Point Operations per Second (FLOPS) regularization. Algorithms such as Bilingual Lexical and Document Expansion Model (BLADE), adopt this approach as a post-processing stage. We propose using Top-P Dynamic Masking similar to Nucleus Sampling in Large Language Models, and demonstrate better performance than Top-K masking. Specifically, we evaluate our methods in the domain of Cross Language Information Retrieval (CLIR)

Figures (3)

• Figure 1: Results of experiment comparing Top-K masking to Top-P-Dynamic masking, re-indexing the small dataset.
• Figure 2: Distributions of terms selected by Top-K masking and Top-P-Dynamic masking for queries and documents, fixing $k=352$ and $p=0.98$. For Top-K masking on documents, the upper limit of 352 terms was chosen for every document.
• Figure 3: The Time Above Average for each Top-P and Top-K Method.