Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Data Attribution in Adaptive Learning

Amit Kiran Rege

Abstract

Machine learning models increasingly generate their own training data -- online bandits, reinforcement learning, and post-training pipelines for language models are leading examples. In these adaptive settings, a single training observation both updates the learner and shifts the distribution of future data the learner will collect. Standard attribution methods, designed for static datasets, ignore this feedback. We formalize occurrence-level attribution for finite-horizon adaptive learning via a conditional interventional target, prove that replay-side information cannot recover it in general, and identify a structural class in which the target is identified from logged data.

Data Attribution in Adaptive Learning

Abstract

Machine learning models increasingly generate their own training data -- online bandits, reinforcement learning, and post-training pipelines for language models are leading examples. In these adaptive settings, a single training observation both updates the learner and shifts the distribution of future data the learner will collect. Standard attribution methods, designed for static datasets, ignore this feedback. We formalize occurrence-level attribution for finite-horizon adaptive learning via a conditional interventional target, prove that replay-side information cannot recover it in general, and identify a structural class in which the target is identified from logged data.

Paper Structure

This paper contains 76 sections, 19 theorems, 204 equations, 1 table.

Table of Contents

  1. Introduction
  2. Related work
  3. A three-axis taxonomy of attribution targets
  4. Problem Formulation
  5. Finite-horizon adaptive learning
  6. Example: Instantiating the Model
  7. Replay and conditional intervention
  8. When recollection collapses to replay
  9. Deconstructing the Adaptive Gap
  10. The Structural Decomposition
  11. The Centered Form
  12. The Insufficiency of Replay-Side Information
  13. Identification under Action-Only Learner State Dependence
  14. Quantifying the Gap between Replay and Intervention
  15. The Directional Failure of Replay
  16. ...and 61 more sections

Key Result

Proposition 1

Fix $t\in\{1,\dots,T\}$ and a realized prefix $h=z_{1:t}\in\mathcal{H}_t$ with $\mathbb P_\nu(h)>0$. Assume that for every future round $s\in\{t+1,\dots,T\}$, every history prefix $z_{1:s-1}\in\mathcal{H}_{s-1}$, and every pair of learner states $\theta,\vartheta\in\Theta$, Then for every $\epsilon\in[-1,\rho]$ and every continuation $c\in\mathcal{C}_t$, Consequently, Hence the finite condition

Theorems & Definitions (38)

  • Proposition 1: Exogenous reduction
  • Theorem 1: Structural decomposition
  • Proposition 2: Centered form of the future-law term
  • Theorem 2: Replay-oracle insufficiency
  • Theorem 3: Exact change of measure in the action-only class
  • Corollary 1: Identification in the action-only class
  • Proposition 3: Unknown context-state dependence destroys identification
  • Theorem 4: Strong separation
  • Theorem 5: Replay in a stable regime
  • Theorem 6: Depth-$L$ recollection identity
  • ...and 28 more