MaxShapley: Towards Incentive-compatible Generative Search with Fair Context Attribution

TL;DR

The paper addresses the challenge of fairly attributing credit to content providers in retrieval-augmented generation (RAG) used by generative search. It introduces MaxShapley, a new attribution method that combines a decomposable max-sum utility with an LLM-as-judge, enabling near-exact Shapley values with a linear number of LLM calls. The core contributions are the new utility function based on key-point decomposition, an efficient per-key-point Shapley computation, and extensive empirical validation on HotPotQA, MuSiQUE, and MS MARCO showing strong attribution quality with substantial cost savings over prior methods. The work also discusses practical reward mechanisms and acknowledges limitations such as LLM evaluation biases and robustness concerns, outlining directions for future improvement and deployment considerations.

Abstract

Generative search engines based on large language models (LLMs) are replacing traditional search, fundamentally changing how information providers are compensated. To sustain this ecosystem, we need fair mechanisms to attribute and compensate content providers based on their contributions to generated answers. We introduce MaxShapley, an efficient algorithm for fair attribution in generative search pipelines that use retrieval-augmented generation (RAG). MaxShapley is a special case of the celebrated Shapley value; it leverages a decomposable max-sum utility function to compute attributions with linear computation in the number of documents, as opposed to the exponential cost of Shapley values. We evaluate MaxShapley on three multi-hop QA datasets (HotPotQA, MuSiQUE, MS MARCO); MaxShapley achieves comparable attribution quality to exact Shapley computation, while consuming a fraction of its tokens--for instance, it gives up to an 8x reduction in resource consumption over prior state-of-the-art methods at the same attribution accuracy.

Figures (11)

• Figure 1: Jaccard index w.r.t. ground truth relevance scores versus token consumption for attribution algorithms on MuSiQUE with GPT4.1o. MaxShapley achieves 0.76 vs. FullShapley's 0.83, while using $6.2\%$ of the token consumption. In contrast, KernelSHAP reaches comparable quality to MaxShapley (0.75) at $8 \times$ the token consumption of MaxShapley.
• Figure 2: System diagram of the attribution problem in RAG pipeline. The query $q$ is used to retrieve a list of information sources $S=\{s_1,\dots,s_m\}$. The Search LLM takes in the query $q$ and the retrieved sources $S$ and generates a concise answer $a$ to the user query. Our goal is to generate a score $\phi_i$ for each information source $s_i$ to quantify its contribution to the final answer $a$.
• Figure 3: Quality of attribution (Jaccard index w.r.t. ground truth (top), Kendall $\tau_b$ w.r.t. FullShapley (bottom)) versus token consumption for attribution algorithms on three datasets, using GPT-4.1o. MaxShapley achieves the same Jaccard index as KernelSHAP with the latter using 8-10$\times$ more tokens. MaxShapley reaches a strong ordinal correlation via Kendall's $\tau_b$ with FullShapley for HotPotQA and MuSiQUE. On MS-MARCO, MaxShapley reaches a moderate ordinal correlation. For similar correlations with FullShapley, KernelSHAP consumes 3-11$\times$ more tokens than MaxShapley.
• Figure 4: Full LLM-as-a-judge prompt, FullShapley and approximation algorithms.
• Figure 5: Full MaxShapley keypoint breakdown prompt.