Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Data Fusion with Distributional Equivalence Test-then-pool

Linying Yang, Xing Liu, Robin J. Evans

Abstract

Randomized controlled trials (RCTs) are the gold standard for causal inference, yet practical constraints often limit the size of the concurrent control arm. Borrowing control data from previous trials offers a potential efficiency gain, but naive borrowing can induce bias when historical and current populations differ. Existing test-then-pool (TTP) procedures address this concern by testing for equality of control outcomes between historical and concurrent trials before borrowing; however, standard implementations may suffer from reduced power or inadequate control of the Type-I error rate. We develop a new TTP framework that fuses control arms while rigorously controlling the Type-I error rate of the final treatment effect test. Our method employs kernel two-sample testing via maximum mean discrepancy (MMD) to capture distributional differences, and equivalence testing to avoid introducing uncontrolled bias, providing a more flexible and informative criterion for pooling. To ensure valid inference, we introduce partial bootstrap and partial permutation procedures for approximating null distributions in the presence of heterogeneous controls. We further establish the overall validity and consistency. We provide empirical studies demonstrating that the proposed approach achieves higher power than standard TTP methods while maintaining nominal error control, highlighting its value as a principled tool for leveraging historical controls in modern clinical trials.

Data Fusion with Distributional Equivalence Test-then-pool

Abstract

Randomized controlled trials (RCTs) are the gold standard for causal inference, yet practical constraints often limit the size of the concurrent control arm. Borrowing control data from previous trials offers a potential efficiency gain, but naive borrowing can induce bias when historical and current populations differ. Existing test-then-pool (TTP) procedures address this concern by testing for equality of control outcomes between historical and concurrent trials before borrowing; however, standard implementations may suffer from reduced power or inadequate control of the Type-I error rate. We develop a new TTP framework that fuses control arms while rigorously controlling the Type-I error rate of the final treatment effect test. Our method employs kernel two-sample testing via maximum mean discrepancy (MMD) to capture distributional differences, and equivalence testing to avoid introducing uncontrolled bias, providing a more flexible and informative criterion for pooling. To ensure valid inference, we introduce partial bootstrap and partial permutation procedures for approximating null distributions in the presence of heterogeneous controls. We further establish the overall validity and consistency. We provide empirical studies demonstrating that the proposed approach achieves higher power than standard TTP methods while maintaining nominal error control, highlighting its value as a principled tool for leveraging historical controls in modern clinical trials.
Paper Structure (44 sections, 16 theorems, 125 equations, 12 figures, 3 tables, 2 algorithms)

This paper contains 44 sections, 16 theorems, 125 equations, 12 figures, 3 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Background
  3. Distributional treatment effect
  4. MMD and two-sample testing
  5. Existing mean-based TTP
  6. Warm-up: Distributional TTP
  7. Method
  8. Fusion test: MMD equivalence testing
  9. Causality test: partial bootstrap and partial permutation
  10. Partial bootstrap
  11. Normal approximation
  12. Partial permutation
  13. Comparison between partial bootstrap and partial permutation
  14. The overall procedure
  15. Choosing the kernel and bandwidth
  16. ...and 29 more sections

Key Result

Theorem 3.1

Let $\alpha_f \in (0, 1)$. Denote by $q_{1-\alpha_f}^{m,\ell}$ the $(1-\alpha_f)$-th quantile of the distribution of $S_{m,\ell} = D(Q_c^m, Q_c) + D(Q_h^l, Q_h)$. Assume $\mathbb{E}_{X, X' \sim Q_c}| k(X, X') |^2 < \infty$ and $\mathbb{E}_{Y, Y' \sim Q_h}| k(Y, Y') |^2 < \infty$. Then there exists $

Figures (12)

  • Figure 1: Geometric explanation of $\cos\beta$. The region with $\cos\beta \leq 0$ is pink.
  • Figure 2: Comparison of the true and approximate distributions of the test statistic under partial bootstrap (first row), partial permutation (middle row) and normal approximation (last row). Results from $1000$ simulations. Within each simulation, $B=1000$ bootstraps/permutations are performed.
  • Figure 3: Type-I error (leftmost panel) and power (rightmost panel) under distributional discrepancy induced by mean shifts. The middle panel shows the proportion of the $1000$ simulations in which historical controls were merged. The parameter $\theta$ is fixed at $0.4$. The RBF kernel and partial bootstrap procedure are used.
  • Figure 4: Type-I error (leftmost panel) and power (rightmost panel) under distributional discrepancy induced by variance shifts. The middle panel shows the proportion of the $1000$ simulations in which historical controls were merged. Type-I error and power results when distribution discrepancy is produced by variance shifts. $\theta$ is fixed as $0.4$. The RBF kernel and partial bootstrap procedure are used.
  • Figure 5: Type-I error (left panel) is controlled across varying $\theta$, despite differing merge rates (right panel). In these experiments, $Q_t = Q_c = \mathcal{N}(0,1)$, $Q_h = \mathcal{N}(\mu_h-\mu_c,1)$. The RBF kernel is used with the partial bootstrap procedure.
  • ...and 7 more figures

Theorems & Definitions (42)

  • Remark 2.1: Connection with ATE
  • Remark 2.3: U-statistics
  • Theorem 3.1: Calibration and consistency of MMD equivalence test
  • Remark 3.3: Generalization of fixed ratios
  • Proposition 3.4: Weak convergence of the partial bootstrap under non-identical control groups
  • Theorem 3.5: Partial bootstrap validity under non-identical control groups
  • Theorem 3.6: Partial bootstrap test consistency
  • Remark 3.7
  • Theorem 3.8: Partial permutation validity
  • Theorem 3.9: Partial permutation consistency
  • ...and 32 more