Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Sea of Words: An In-Depth Analysis of Anchors for Text Data

Gianluigi Lopardo, Frederic Precioso, Damien Garreau

TL;DR

This work provides the first theoretical analysis of Anchors for text data by formalizing an exhaustive p-Anchors framework under TF-IDF vectorization and UNK-based word removal. It proves sampling equivalence to independent word removals, establishes stability of exhaustive anchors under perturbations of the evaluation function, and characterizes anchor selection for simple if-then rules and linear classifiers, showing that anchors tend to pick words with positive influence weighted by IDF. For neural networks, it empirically links anchor selections to the words corresponding to the largest input gradients scaled by IDF, suggesting a fast gradient-based proxy for explanations when model access is available. The results offer solid theoretical guarantees for local rule-based explainers, illuminate the behavior of Anchors on different model classes, and provide a practical bridge between model-agnostic explanations and gradient-based interpretations. Collectively, the framework aids the principled design and analysis of interpretability methods in NLP and beyond.

Abstract

Anchors (Ribeiro et al., 2018) is a post-hoc, rule-based interpretability method. For text data, it proposes to explain a decision by highlighting a small set of words (an anchor) such that the model to explain has similar outputs when they are present in a document. In this paper, we present the first theoretical analysis of Anchors, considering that the search for the best anchor is exhaustive. After formalizing the algorithm for text classification, we present explicit results on different classes of models when the vectorization step is TF-IDF, and words are replaced by a fixed out-of-dictionary token when removed. Our inquiry covers models such as elementary if-then rules and linear classifiers. We then leverage this analysis to gain insights on the behavior of Anchors for any differentiable classifiers. For neural networks, we empirically show that the words corresponding to the highest partial derivatives of the model with respect to the input, reweighted by the inverse document frequencies, are selected by Anchors.

A Sea of Words: An In-Depth Analysis of Anchors for Text Data

TL;DR

This work provides the first theoretical analysis of Anchors for text data by formalizing an exhaustive p-Anchors framework under TF-IDF vectorization and UNK-based word removal. It proves sampling equivalence to independent word removals, establishes stability of exhaustive anchors under perturbations of the evaluation function, and characterizes anchor selection for simple if-then rules and linear classifiers, showing that anchors tend to pick words with positive influence weighted by IDF. For neural networks, it empirically links anchor selections to the words corresponding to the largest input gradients scaled by IDF, suggesting a fast gradient-based proxy for explanations when model access is available. The results offer solid theoretical guarantees for local rule-based explainers, illuminate the behavior of Anchors on different model classes, and provide a practical bridge between model-agnostic explanations and gradient-based interpretations. Collectively, the framework aids the principled design and analysis of interpretability methods in NLP and beyond.

Abstract

Anchors (Ribeiro et al., 2018) is a post-hoc, rule-based interpretability method. For text data, it proposes to explain a decision by highlighting a small set of words (an anchor) such that the model to explain has similar outputs when they are present in a document. In this paper, we present the first theoretical analysis of Anchors, considering that the search for the best anchor is exhaustive. After formalizing the algorithm for text classification, we present explicit results on different classes of models when the vectorization step is TF-IDF, and words are replaced by a fixed out-of-dictionary token when removed. Our inquiry covers models such as elementary if-then rules and linear classifiers. We then leverage this analysis to gain insights on the behavior of Anchors for any differentiable classifiers. For neural networks, we empirically show that the words corresponding to the highest partial derivatives of the model with respect to the input, reweighted by the inverse document frequencies, are selected by Anchors.
Paper Structure (48 sections, 19 theorems, 87 equations, 16 figures, 3 tables, 1 algorithm)

This paper contains 48 sections, 19 theorems, 87 equations, 16 figures, 3 tables, 1 algorithm.

Table of Contents

  1. INTRODUCTION
  2. Contributions.
  3. Related work.
  4. ANCHORS FOR TEXT DATA
  5. Setting and notation
  6. Precision and coverage
  7. The algorithm
  8. The sampling
  9. EXHAUSTIVE p-ANCHORS
  10. Description of the algorithm
  11. Stability with respect to the evaluation function
  12. ANALYSIS ON EXPLAINABLE CLASSIFIERS
  13. Vectorizers and immediate consequences
  14. Simple if-then rules
  15. Linear classifiers
  16. ...and 33 more sections

Key Result

Proposition 1

The sampling process described above is equivalent to replacing, for any sample $x_i$, each word $x_{i,k}$ such that $k\notin A$ independently with probability $1/2$.

Figures (16)

  • Figure 1: Anchors explaining the positive prediction of a black-box model $f$ on an example $\xi$ from the Restaurant review dataset. The anchor $A = \{\textit{great, not, bad, fine}\}$ (in blue), having length $\left\lvert A\right\rvert = 4$ is selected. Intuitively, these four words together ensure a positive prediction by $f$ with high probability ($\texttt{precision}: \; 0.97$), while being not too uncommon ($\texttt{coverage}: \; 0.12$).
  • Figure 2: Anchors' sampling is performed in three main steps: copies generation, random selection, word replacing. Here, for instance, for the fourth word ("fox"), $B_4=2$ and $S_4=\{2,n\}$, so the second and the $n$-th copies are considered for replacement.
  • Figure 3: An illustration of Algorithm \ref{['algo:exhaustive-anchors']} with evaluation function $p=\mathrm{Prec}$. Each blue dot is an anchor, with $x$ coordinate its length and $y$ coordinate its value for $p$. Here, $\varepsilon=0.2$ and the maximal length of an anchor is $b=10$ (the length of $\xi$). In the end, the anchor $A$ such that $\left\lvert A\right\rvert=3$ and $p(A)=0.9$ is selected (red circle).
  • Figure 4: Making a word disappear from the explanation by adding one occurrence. The model predicts $1$ if "very" and "good" are present. The count corresponds to the appearance of the word(s) in the selected anchor on $100$ runs of Anchors. When the multiplicity of "very" in the document crosses the breakpoint value $B=4$ (by default, $\varepsilon=0.05$), it disappears from the selected anchor with high probability (on the top panel, $m_{\text{very}}=4$, on the bottom panel, $m_{\text{very}}=5$).
  • Figure 5: Illustration of Proposition \ref{['prop:approx-prec-maximization']}. On linear models, the algorithm includes words having the highest $\lambda_jv_j$s first. Finally, the minimal anchor satisfying the precision condition $\overline{\Phi}\left(L\left(A\right)\right) \approx\mathrm{Prec}(A) \geq 1-\varepsilon$ is selected, which is $A=(m_1,m_2,m_3,2,0,\ldots,0,0)$ in the example.
  • ...and 11 more figures

Theorems & Definitions (32)

  • Proposition 1: Equivalent sampling
  • Proposition 2: Stability of exhaustive p-Anchors
  • Proposition 3: $\widehat{\mathrm{Prec}}_n(A)$ uniformly approximates $\mathrm{Prec}$
  • Definition 1: TF-IDF
  • Proposition 4: Dummy features
  • Proposition 5: Presence of a set of words
  • Proposition 6: Precision of a linear classifier
  • Proposition 7: Approximate precision maximization
  • Claim 1
  • Claim 2
  • ...and 22 more