Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Continuous time asymptotic representations for adaptive experiments

Karun Adusumilli

Abstract

This article develops a continuous-time asymptotic framework for analyzing adaptive experiments -- settings in which data collection and treatment assignment evolve dynamically in response to incoming information. A key challenge in analyzing fully adaptive experiments, where the assignment policy is updated after each observation, is that the sequence of policy rules often lack a well-defined asymptotic limit. To address this, we focus instead on the empirical allocation process, which captures the fraction of observations assigned to each treatment over time. We show that, under general conditions, any adaptive experiment and its associated empirical allocation process can be approximated by a limit experiment defined by Gaussian diffusions with unknown drifts and a corresponding continuous-time allocation process. This limit representation facilitates the analysis of optimal decision rules by reducing the dimensionality of the state-space and leveraging the tractability of Gaussian diffusions. We apply the framework to derive optimal estimators, analyze in-sample regret for adaptive experiments, and construct e-processes for anytime-valid inference. Notably, we introduce the first definition of any-time and any-experiment valid inference for multi-treatment settings.

Continuous time asymptotic representations for adaptive experiments

Abstract

This article develops a continuous-time asymptotic framework for analyzing adaptive experiments -- settings in which data collection and treatment assignment evolve dynamically in response to incoming information. A key challenge in analyzing fully adaptive experiments, where the assignment policy is updated after each observation, is that the sequence of policy rules often lack a well-defined asymptotic limit. To address this, we focus instead on the empirical allocation process, which captures the fraction of observations assigned to each treatment over time. We show that, under general conditions, any adaptive experiment and its associated empirical allocation process can be approximated by a limit experiment defined by Gaussian diffusions with unknown drifts and a corresponding continuous-time allocation process. This limit representation facilitates the analysis of optimal decision rules by reducing the dimensionality of the state-space and leveraging the tractability of Gaussian diffusions. We apply the framework to derive optimal estimators, analyze in-sample regret for adaptive experiments, and construct e-processes for anytime-valid inference. Notably, we introduce the first definition of any-time and any-experiment valid inference for multi-treatment settings.
Paper Structure (49 sections, 11 theorems, 158 equations, 6 figures)

This paper contains 49 sections, 11 theorems, 158 equations, 6 figures.

Table of Contents

  1. Introduction
  2. Related literature
  3. Adaptive experiments, Policy rules and Allocation processes
  4. An illustrative example
  5. General setup
  6. Empirical allocation processes
  7. Local asymptotics
  8. Score processes
  9. The limit experiment
  10. Sufficient statistics
  11. Allocation processes vs policy rules
  12. Asymptotic equivalence of experiments
  13. Convergence of likelihood ratio processes
  14. The main result
  15. Behavior under local alternatives
  16. ...and 34 more sections

Key Result

Lemma 1

Under $\bm{h}=(0,0)$, the processes $x_{1}(\cdot),x_{0}(\cdot)$ are $\mathcal{I}_{t}$-martingales with quadratic variations $q_{1}(t),q_{0}(t)$.

Figures (6)

  • Figure 2.1: Distribution of $q_{n,1}(t)$ under Thompson Sampling
  • Figure 2.2: Distribution of $q_{n,1}(t)$ under UCB
  • Figure 4.1: Point estimation: two-armed UCB
  • Figure 4.2: Point estimation: two-armed Thompson Sampling
  • Figure 6.1: Anytime-valid inference
  • ...and 1 more figures

Theorems & Definitions (16)

  • Definition 1
  • Lemma 1
  • Theorem 1
  • Corollary 1
  • Theorem 2
  • Corollary 2
  • Theorem 3
  • Definition 2
  • Definition 3
  • Theorem 4
  • ...and 6 more