Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Influence Dynamics and Stagewise Data Attribution

Jin Hwa Lee, Matthew Smith, Maxwell Adam, Jesse Hoogland

TL;DR

This work critiques static training data attribution in neural networks and introduces a stagewise data attribution framework grounded in Singular Learning Theory (SLT). By shifting from classical, Hessian-based influence functions to Bayesian Influence Functions (BIF), the authors predict and observe non-monotonic influence dynamics, including sign flips and peaks at developmental phase transitions. They validate these predictions analytically and in a toy hierarchical model, showing that shifts in influence track the progressive learning of semantic structure, and extend the analysis to large language models where token-level influence aligns with known developmental stages such as induction circuits. The results offer a new lens for developmental interpretability and implicit curricula, while outlining limitations and future work to connect Bayesian SLT more directly with non-equilibrium SGD dynamics and mechanistic interpretability.

Abstract

Current training data attribution (TDA) methods treat the influence one sample has on another as static, but neural networks learn in distinct stages that exhibit changing patterns of influence. In this work, we introduce a framework for stagewise data attribution grounded in singular learning theory. We predict that influence can change non-monotonically, including sign flips and sharp peaks at developmental transitions. We first validate these predictions analytically and empirically in a toy model, showing that dynamic shifts in influence directly map to the model's progressive learning of a semantic hierarchy. Finally, we demonstrate these phenomena at scale in language models, where token-level influence changes align with known developmental stages.

Influence Dynamics and Stagewise Data Attribution

TL;DR

This work critiques static training data attribution in neural networks and introduces a stagewise data attribution framework grounded in Singular Learning Theory (SLT). By shifting from classical, Hessian-based influence functions to Bayesian Influence Functions (BIF), the authors predict and observe non-monotonic influence dynamics, including sign flips and peaks at developmental phase transitions. They validate these predictions analytically and in a toy hierarchical model, showing that shifts in influence track the progressive learning of semantic structure, and extend the analysis to large language models where token-level influence aligns with known developmental stages such as induction circuits. The results offer a new lens for developmental interpretability and implicit curricula, while outlining limitations and future work to connect Bayesian SLT more directly with non-equilibrium SGD dynamics and mechanistic interpretability.

Abstract

Current training data attribution (TDA) methods treat the influence one sample has on another as static, but neural networks learn in distinct stages that exhibit changing patterns of influence. In this work, we introduce a framework for stagewise data attribution grounded in singular learning theory. We predict that influence can change non-monotonically, including sign flips and sharp peaks at developmental transitions. We first validate these predictions analytically and empirically in a toy model, showing that dynamic shifts in influence directly map to the model's progressive learning of a semantic hierarchy. Finally, we demonstrate these phenomena at scale in language models, where token-level influence changes align with known developmental stages.
Paper Structure (64 sections, 53 equations, 15 figures, 2 tables, 1 algorithm)

This paper contains 64 sections, 53 equations, 15 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Contributions.
  3. Theory
  4. From Uniform to Stagewise Development
  5. Developmental Interpretability: from SGD to Bayes and back again.
  6. The regular learning process (Bernstein--von Mises).
  7. The singular learning process (Watanabe).
  8. From Static to Developmental Influence Functions
  9. Classical influence functions: a static view.
  10. Bayesian influence functions: a developmental tool.
  11. Estimating (local) Bayesian influence functions.
  12. Stagewise Data Attribution
  13. Influence and susceptibility.
  14. Setup: a bimodal posterior.
  15. Setup: a bimodal posterior.
  16. ...and 49 more sections

Figures (15)

  • Figure 1: Stagewise learning requires stagewise data attribution. (a) In regular models, development is a uniform, monotonic process of posterior concentration around a single solution (Bernstein--von Mises). (b) In singular models, development is a stagewise process where the posterior undergoes phase transitions (Watanabe's singular learning theory). This stagewise development means the influence one sample has on another can profoundly change over the course of the learning process.
  • Figure 2: A toy model of hierarchical semantic knowledge acquisition. (a) A toy dataset adopted from saxe2019mathematical. Each object maps to a feature vector that describes the hierarchical structure of semantic knowledge, e.g., 'penguin' and cat are all living animals, but 'penguin' is a bird and cat is a mammal. (b) The correlation matrix of the feature output shows a hierarchical structure. (c) The hierarchical structure is acquired progressively during the training of the deep linear neural network.
  • Figure 3: Influence over time on a hierarchical semantic dataset. We measure the influence of dog sample on other query samples $j$ with the following: (Left) BIF ($\beta = 1000, \epsilon=1e\text{-}3, \gamma = 5e+3$). (Center) Analytical IF (see full derivation in \ref{['app:toy-analytic']}). (Right) Loss difference from Leave-One-Out (LOO) retraining experiment. All three measures agree that the influence one sample has on another can vary non-monotonically over the course of training as discussed in \ref{['sec:toy-model']}. See \ref{['appendix:toy-model-bif-hp']} for additional pairs of samples and experimental details.
  • Figure 4: BIF captures developmental influence. (a) The peak positive influence from dog to different data points is noted with ①, ②, and ③. (b) MDS of the hidden representations in the network over the course of learning. The peaks in influence match the branching points of the MDS trajectory, where each hierarchical category develops (black points).
  • Figure 5: Token--class relationships. BIF between structural classes through training. We observe structural relationships between tokens reflected in influence patterns between classes, including strong intra-class relationships, development of induction, and relationships between word elements and corresponding delimiter token classes. Dashed lines indicate major inflection points in the BIF. Opacity captures strongly influencing pairs of tokens.
  • ...and 10 more figures