Compact Example-Based Explanations for Language Models

TL;DR

This work tackles how to present interpretable, example-based explanations for language models by focusing not only on influence estimates but crucially on how a selected set of training examples supports a given test instance. It introduces the selection relevance score $\xi^{\mathcal{SR}}$, a retraining-free metric that evaluates how well the gradients of selected training examples can reconstruct the test-instance gradient, thereby measuring local relevance and reducing redundancy. Through gradient-based encoding, constrained scoring models, and fine-tuning-based validation, the authors show that naive high-influence selections often underperform random or coverage-aware strategies, and they propose a new approach that balances influence with representativeness to improve selection budgets. The results suggest that the choice of selection strategy substantially impacts explanation usefulness, and that selection relevance can predict favorable training dynamics, motivating explicit consideration of selection relevance in explanation systems.

Abstract

Training data influence estimation methods quantify the contribution of training documents to a model's output, making them a promising source of information for example-based explanations. As humans cannot interpret thousands of documents, only a small subset of the training data can be presented as an explanation. Although the choice of which documents to include directly affects explanation quality, previous evaluations of such systems have largely ignored any selection strategies. To address this, we propose a novel selection relevance score, a retraining-free metric that quantifies how useful a set of examples is for explaining a model's output. We validate this score through fine-tuning experiments, confirming that it can predict whether a set of examples supports or undermines the model's predictions. Using this metric, we further show that common selection strategies often underperform random selection. Motivated by this finding, we propose a strategy that balances influence and representativeness, enabling better use of selection budgets than naively selecting the highest-ranking examples.

Figures (8)

• Figure 1: Simplified scoring setup. We score example-based explanations based on how well the test instance's loss gradient (left) can be reconstructed (right) as a linear combination of the selected examples' gradients.
• Figure 2: Case Study. t-SNE visualizations of the 100 most-influential examples by DataInf in gradient and embedding spaces (https://huggingface.co/Qwen/Qwen3-Embedding-0.6B: zhang_qwen3_2025). Our selections (FL) improve coverage.
• Figure 3: Case Study. First 3 documents per selection ($k$=5, DataInf, most inf.). Full figure in Appendix \ref{['appendix:case_study']}.
• Figure 4: Improvement over naive selection. Relative increase over naive selection in per-cent and dB. Green: our facility location-based selections at different $\lambda$.
• Figure 5: Fine-tuning based validation. Relationship between prediction support $\xi^{+}$ and selection relevance.