AGRaME: Any-Granularity Ranking with Multi-Vector Embeddings

TL;DR

The paper tackles the need for flexible ranking at multiple granularity levels while keeping a single coarse encoding. It introduces AGRaME, a framework that leverages multi-vector embeddings to enable ranking from coarse passages down to finer subunits, guided by a multi-granular contrastive training objective. The approach yields notable gains in sentence-level ranking without sacrificing passage-level performance and extends to proposition-level attribution with a practical post-hoc citation method, PropCite, that improves retrieval-augmented generation. Overall, AGRaME enables more precise retrieval for open-domain QA and attribution tasks, with direct applications in post-hoc citation and evidence gathering for generated content.

Abstract

Ranking is a fundamental and popular problem in search. However, existing ranking algorithms usually restrict the granularity of ranking to full passages or require a specific dense index for each desired level of granularity. Such lack of flexibility in granularity negatively affects many applications that can benefit from more granular ranking, such as sentence-level ranking for open-domain question-answering, or proposition-level ranking for attribution. In this work, we introduce the idea of any-granularity ranking, which leverages multi-vector embeddings to rank at varying levels of granularity while maintaining encoding at a single (coarser) level of granularity. We propose a multi-granular contrastive loss for training multi-vector approaches, and validate its utility with both sentences and propositions as ranking units. Finally, we demonstrate the application of proposition-level ranking to post-hoc citation addition in retrieval-augmented generation, surpassing the performance of prompt-driven citation generation.

Figures (4)

• Figure 1: Ranking at different levels of granularity. X$\rightarrow$Y is used to denote that X represents the query granularity used for ranking, with entire query encoded, and Y indicates the granularity of the retrieval unit being ranked, with entire retrieval unit encoded. In addition to the typical ranking setting (A), our proposed approach enables ranking finer retrieval units (B and D) or using finer query units for ranking (C and D).
• Figure 2: Figure demonstrating our sentence-level scoring methodology using multi-vector representations with encoding at passage-level. Query marker $m_q$ is used while getting passage-level score $P$, while marker $m'_q$ is used for getting sentence-level scores $S1$, $S2$, $S3$.
• Figure 3: Comparison of training curves for sentence-level and passage-level loss, when a different query marker is used. The model converges faster at sentence-level with a different query marker, while passage-level loss is mostly similar for the two.
• Figure 4: Figure illustrating PropCite, our proposed approach for post-hoc addition of citations to long-form answers. PropCite encodes sentences and uses the propositions within them as queries for attribution. The figure shows the propositions highlighted within the current sentence (in yellow), and the corresponding supporting evidence highlighted in the input context passages. PropCite correctly attributes proposition P2 to context C1, while directly encoding and querying using P2 incorrectly attributes to C2.