Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Testing Random Effects for Binomial Data

Lucas Kania, Larry Wasserman, Sivaraman Balakrishnan

TL;DR

This work develops a locality-aware, minimax framework for goodness-of-fit and homogeneity testing of random effects in binomial data, using the 1-Wasserstein distance to compare mixing distributions. The authors introduce two complementary tests—the plug-in $W_1$ test and a debiased Pearson's $\chi^2$ test based on Kravchuk polynomials—and show they are minimax-optimal up to constants, with explicit separation rates that depend on $n$ and $t$. They extend the analysis to both reference-driven and reference-free homogeneity problems, deriving reductions to fixed-effects testing and constructing debiased Cochran's $\chi^2$ variants for unknown-null settings. The methods are demonstrated in simulations and applied to a rosiglitazone meta-analysis, yielding tighter inference and actionable conclusions about cross-study homogeneity and model selection in meta-analytic practice.

Abstract

In modern scientific research, small-scale studies with limited participants are increasingly common. However, interpreting individual outcomes can be challenging, making it standard practice to combine data across studies using random effects to draw broader scientific conclusions. In this work, we introduce an optimal methodology for assessing the goodness of fit of a reference distribution for the random effects arising from binomial counts. For meta-analyses, we also derive optimal tests to evaluate whether multiple studies are in agreement before pooling the data. In all cases, we prove that the proposed tests optimally distinguish null and alternative hypotheses separated in the 1-Wasserstein distance.

Testing Random Effects for Binomial Data

TL;DR

This work develops a locality-aware, minimax framework for goodness-of-fit and homogeneity testing of random effects in binomial data, using the 1-Wasserstein distance to compare mixing distributions. The authors introduce two complementary tests—the plug-in test and a debiased Pearson's test based on Kravchuk polynomials—and show they are minimax-optimal up to constants, with explicit separation rates that depend on and . They extend the analysis to both reference-driven and reference-free homogeneity problems, deriving reductions to fixed-effects testing and constructing debiased Cochran's variants for unknown-null settings. The methods are demonstrated in simulations and applied to a rosiglitazone meta-analysis, yielding tighter inference and actionable conclusions about cross-study homogeneity and model selection in meta-analytic practice.

Abstract

In modern scientific research, small-scale studies with limited participants are increasingly common. However, interpreting individual outcomes can be challenging, making it standard practice to combine data across studies using random effects to draw broader scientific conclusions. In this work, we introduce an optimal methodology for assessing the goodness of fit of a reference distribution for the random effects arising from binomial counts. For meta-analyses, we also derive optimal tests to evaluate whether multiple studies are in agreement before pooling the data. In all cases, we prove that the proposed tests optimally distinguish null and alternative hypotheses separated in the 1-Wasserstein distance.

Paper Structure

This paper contains 69 sections, 48 theorems, 355 equations, 16 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Summary of main contributions
  3. The minimax framework
  4. Goodness-of-fit testing
  5. The plugin test
  6. The debiased Pearson's chi-squared test
  7. Lower bounds on the critical separation
  8. Simulations
  9. Homogeneity testing with a reference effect
  10. Reduction from random to fixed effects
  11. Lower bound on the critical separation
  12. Homogeneity testing without a reference effect
  13. A conservative test
  14. The debiased Cochran's chi-squared test
  15. Application to meta-analyses and model selection
  16. ...and 54 more sections

Key Result

Theorem 1

For hypotheses eq:w1_testing, the plug-in test eq:w1_plugin_test controls the type I error by $\alpha$. Moreover, there exists a universal positive constant $C$ such that the test controls the type II error by $\beta$ whenever

Figures (16)

  • Figure 1: Power curves for valid tests under alternative distribution generated by \ref{['eq:low_prob_perturbation']}. All tests have the same power curves regardless of the number of trials $t$.
  • Figure 2: The left panel displays the Wasserstein distance between the moment matching distributions as the number of matched moments increased. The right panel displays two distributions that match 10 moments.
  • Figure 3: Power curves for valid tests under alternatives generated by lemma:moment_matching_distributions.
  • Figure 4: Empirical critical separation as the number of trials $t$ is varied below and above the number of observations $n$
  • Figure 5: Histograms of proportions of the proportion of patients that suffered a myocardial infarction and died from cardiovascular causes due to the application of the rosiglitazone treatment for the treatment and control groups.
  • ...and 11 more figures

Theorems & Definitions (84)

  • Theorem 1
  • Lemma 1: Theorem 3.2 of balakrishnanHypothesisTestingDensities2019
  • Theorem 2
  • Lemma 2: Le Cam's method, Theorem 2.2 of tsybakovIntroductionNonparametricEstimation2009
  • Lemma 3
  • Lemma 4: Appendix E of wuMinimaxRatesEntropy2016
  • Theorem 3
  • Proposition 1: Proposition 2 of kong2017
  • Lemma 5
  • Lemma 6
  • ...and 74 more