IMRNNs: An Efficient Method for Interpretable Dense Retrieval via Embedding Modulation
Yash Saxena, Ankur Padia, Kalpa Gunaratna, Manas Gaur
TL;DR
Dense retrievers suffer from static embeddings that hinder query–document adaptation and obscured interpretability. IMRNNs address this by adding two lightweight adapters that perform bidirectional modulation in a low-dimensional space, yielding interpretable changes to embeddings: $\mathbf{d}_{\text{mod}} = \mathbf{W}_q \mathbf{d}_{\text{proj}} + \mathbf{b}_q$ and $\mathbf{q}_{\text{mod}} = \bar{\mathbf{W}}_d \mathbf{q}_{\text{proj}} + \bar{\mathbf{b}}_d$, with final similarity $\text{score} = \cos(\mathbf{q}_{\text{mod}}, \mathbf{d}_{\text{mod}})$. A margin-based ranking objective trains only the adapters while the base encoder remains frozen, and interpretability is achieved via Moore–Penrose back-projection to connect embedding changes to human-understandable tokens. Empirically, IMRNNs yield consistent retrieval improvements across seven BEIR datasets and generalize across multiple base retrievers, demonstrating that bidirectional modulation can explain and enhance retrieval without re-encoding. The approach remains plug-and-play, offering three levels of interpretability and enabling domain-aware semantic emphasis, with limitations including potential token-noise in attribution and higher inference latency in large corpora.
Abstract
Interpretability in black-box dense retrievers remains a central challenge in Retrieval-Augmented Generation (RAG). Understanding how queries and documents semantically interact is critical for diagnosing retrieval behavior and improving model design. However, existing dense retrievers rely on static embeddings for both queries and documents, which obscures this bidirectional relationship. Post-hoc approaches such as re-rankers are computationally expensive, add inference latency, and still fail to reveal the underlying semantic alignment. To address these limitations, we propose Interpretable Modular Retrieval Neural Networks (IMRNNs), a lightweight framework that augments any dense retriever with dynamic, bidirectional modulation at inference time. IMRNNs employ two independent adapters: one conditions document embeddings on the current query, while the other refines the query embedding using corpus-level feedback from initially retrieved documents. This iterative modulation process enables the model to adapt representations dynamically and expose interpretable semantic dependencies between queries and documents. Empirically, IMRNNs not only enhance interpretability but also improve retrieval effectiveness. Across seven benchmark datasets, applying our method to standard dense retrievers yields average gains of +6.35% nDCG, +7.14% recall, and +7.04% MRR over state-of-the-art baselines. These results demonstrate that incorporating interpretability-driven modulation can both explain and enhance retrieval in RAG systems.