RAGE Against the Machine: Retrieval-Augmented LLM Explanations

TL;DR

RAGE advances explainability for retrieval-augmented LLMs by computing provenance through counterfactual context perturbations. It formalizes Answer Origin Explainability via context combinations and permutations, and implements pruning and scoring to make the search tractable. The system combines a Dash-based UI, local BM25 retrieval, and a 7B-Llama-2 chat model to analyze how source provenance shapes answers, using $a = L(q, D_q)$ and $S(q, d, D_q)$ to quantify influence. Efficient algorithms—Fisher-Yates sampling for permutations and an assignment-problem-based method for optimal permutations—enable scalable exploration across multiple use cases, with interactive visualizations that reveal which sources and positions drive outputs.

Abstract

This paper demonstrates RAGE, an interactive tool for explaining Large Language Models (LLMs) augmented with retrieval capabilities; i.e., able to query external sources and pull relevant information into their input context. Our explanations are counterfactual in the sense that they identify parts of the input context that, when removed, change the answer to the question posed to the LLM. RAGE includes pruning methods to navigate the vast space of possible explanations, allowing users to view the provenance of the produced answers.

Figures (2)

• Figure 1: The architecture of $\sf{RAGE}$.
• Figure 2: Combination insights for the query about The Big Three.