Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Efficiently Computing Susceptibility to Context in Language Models

Tianyu Liu, Kevin Du, Mrinmaya Sachan, Ryan Cotterell

TL;DR

Fisher susceptibility is proposed, an efficient method to estimate the susceptibility of language models based on Fisher information that is comparable to Monte Carlo estimated susceptibility across a diverse set of query domains despite its being $70\times faster.

Abstract

One strength of modern language models is their ability to incorporate information from a user-input context when answering queries. However, they are not equally sensitive to the subtle changes to that context. To quantify this, Du et al. (2024) gives an information-theoretic metric to measure such sensitivity. Their metric, susceptibility, is defined as the degree to which contexts can influence a model's response to a query at a distributional level. However, exactly computing susceptibility is difficult and, thus, Du et al. (2024) falls back on a Monte Carlo approximation. Due to the large number of samples required, the Monte Carlo approximation is inefficient in practice. As a faster alternative, we propose Fisher susceptibility, an efficient method to estimate the susceptibility based on Fisher information. Empirically, we validate that Fisher susceptibility is comparable to Monte Carlo estimated susceptibility across a diverse set of query domains despite its being $70\times$ faster. Exploiting the improved efficiency, we apply Fisher susceptibility to analyze factors affecting the susceptibility of language models. We observe that larger models are as susceptible as smaller ones.

Efficiently Computing Susceptibility to Context in Language Models

TL;DR

Fisher susceptibility is proposed, an efficient method to estimate the susceptibility of language models based on Fisher information that is comparable to Monte Carlo estimated susceptibility across a diverse set of query domains despite its being $70\times faster.

Abstract

One strength of modern language models is their ability to incorporate information from a user-input context when answering queries. However, they are not equally sensitive to the subtle changes to that context. To quantify this, Du et al. (2024) gives an information-theoretic metric to measure such sensitivity. Their metric, susceptibility, is defined as the degree to which contexts can influence a model's response to a query at a distributional level. However, exactly computing susceptibility is difficult and, thus, Du et al. (2024) falls back on a Monte Carlo approximation. Due to the large number of samples required, the Monte Carlo approximation is inefficient in practice. As a faster alternative, we propose Fisher susceptibility, an efficient method to estimate the susceptibility based on Fisher information. Empirically, we validate that Fisher susceptibility is comparable to Monte Carlo estimated susceptibility across a diverse set of query domains despite its being faster. Exploiting the improved efficiency, we apply Fisher susceptibility to analyze factors affecting the susceptibility of language models. We observe that larger models are as susceptible as smaller ones.

Paper Structure

This paper contains 26 sections, 1 theorem, 13 equations, 6 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Susceptibility to Context in LMs
  3. Measuring Susceptibility
  4. Susceptibility as Mutual Information
  5. Computational Cost
  6. Fisher Susceptibility
  7. A Simple Reparameterization
  8. The Fisher Information Matrix
  9. Fisher and Mutual Information
  10. Efficient Computation
  11. Experiments
  12. Experiment Setup
  13. Dataset.
  14. Models.
  15. Computational Resources.
  16. ...and 11 more sections

Key Result

Proposition A.1

where $\boldsymbol{\delta}({\color{MyOrange}{c}}, {\color{MyGreen}{q}}) \triangleq \boldsymbol{\theta}({\color{MyOrange}{c}} \oplus {\color{MyGreen}{q}}) - \boldsymbol{\theta}({\color{MyGreen}{q}})$.

Figures (6)

  • Figure 1: This plot shows the average susceptibility on 122 relation domains (e.g., alumniOf) in the YAGO dataset. Each point represents the average score of queries on a relation domain. The $x$-coordinate represents Monte Carlo susceptibility and $y$-coordinate represents Fisher susceptibility. The scores are computed with LLaMA-3-8B-Instruct. For both query types, the two metrics are strongly correlated.
  • Figure 2: For both open and closed queries, Fisher susceptibility $y$ strongly correlates with Monte Carlo susceptibility ($x$-axis). Monte Carlo susceptibility is divided into 5 bins from $0$ to $1$ on LLaMA-3-8B-Instruct.
  • Figure 3: Susceptibility comparisons between open and closed queries for real entities (bars on the left) and fake entities (bars on the right). In each subplot, the two bars on the left represent open queries, and the two on the right represent closed queries. From this, we can see the susceptibility generally does not appear to differ much between real and fake entities. (Top) Monte Carlo susceptibility. (Bottom) Fisher susceptibility.
  • Figure 4: Susceptibility comparisons between (a) question-answering and (b) sentence-completion queries. Consistently, question-answering formats are less susceptible for both susceptibility metrics. (Top) Monte Carlo susceptibility. (Bottom) Fisher susceptibility.
  • Figure 5: Fisher susceptibility plotted against Monte Carlo susceptibility for different models.
  • ...and 1 more figures

Theorems & Definitions (2)

  • Proposition A.1
  • proof