Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On the statistical analysis of grouped data: when Pearson $χ^2$ and other divisible statistics are not goodness-of-fit tests

Sara Algeri, Estate V. Khmaladze

TL;DR

This work develops a unifying, linear-functional framework for divisible statistics in the analysis of grouped Poisson data, addressing the common practice that treats Pearson's χ^2 as a GOF benchmark. By casting many test statistics as linear functionals of a random measure and allowing parameter estimation, the authors derive projection-based representations that reveal when tests gain or lose power under contiguous alternatives. A key finding is that, in sparse regimes, no single divisible statistic is adequate for GOF; instead, power is best achieved through ensembles of partial-sums tests and optimal weighting, with parameter estimation handled via projection and a projected bootstrap. The paper further provides practically applicable methods, including an omnibus KS-type test based on partial sums and a distribution-free variant via unitary transformations, and demonstrates these methods on Chandra X-ray background data, highlighting the practical impact for physics and astronomy analyses.

Abstract

Thousands of experiments are analyzed and papers are published each year involving the statistical analysis of grouped data. While this area of statistics is often perceived -- somewhat naively -- as saturated, several misconceptions still affect everyday practice, and new frontiers have so far remained unexplored. Researchers must be aware of the limitations affecting their analyses and what are the new possibilities in their hands. Motivated by this need, the article introduces a unifying approach to the analysis of grouped data, which allows us to study the class of divisible statistics -- that includes Pearson's $χ^2$, the likelihood ratio as special cases -- with a fresh perspective. The contributions collected in this manuscript span from modeling and estimation to distribution-free goodness-of-fit tests. Perhaps the most surprising result presented here is that, in a sparse regime, all tests proposed in the literature are dominated by members of the class of weighted linear statistics.

On the statistical analysis of grouped data: when Pearson $χ^2$ and other divisible statistics are not goodness-of-fit tests

TL;DR

This work develops a unifying, linear-functional framework for divisible statistics in the analysis of grouped Poisson data, addressing the common practice that treats Pearson's χ^2 as a GOF benchmark. By casting many test statistics as linear functionals of a random measure and allowing parameter estimation, the authors derive projection-based representations that reveal when tests gain or lose power under contiguous alternatives. A key finding is that, in sparse regimes, no single divisible statistic is adequate for GOF; instead, power is best achieved through ensembles of partial-sums tests and optimal weighting, with parameter estimation handled via projection and a projected bootstrap. The paper further provides practically applicable methods, including an omnibus KS-type test based on partial sums and a distribution-free variant via unitary transformations, and demonstrates these methods on Chandra X-ray background data, highlighting the practical impact for physics and astronomy analyses.

Abstract

Thousands of experiments are analyzed and papers are published each year involving the statistical analysis of grouped data. While this area of statistics is often perceived -- somewhat naively -- as saturated, several misconceptions still affect everyday practice, and new frontiers have so far remained unexplored. Researchers must be aware of the limitations affecting their analyses and what are the new possibilities in their hands. Motivated by this need, the article introduces a unifying approach to the analysis of grouped data, which allows us to study the class of divisible statistics -- that includes Pearson's , the likelihood ratio as special cases -- with a fresh perspective. The contributions collected in this manuscript span from modeling and estimation to distribution-free goodness-of-fit tests. Perhaps the most surprising result presented here is that, in a sparse regime, all tests proposed in the literature are dominated by members of the class of weighted linear statistics.
Paper Structure (19 sections, 6 theorems, 113 equations, 4 figures)

This paper contains 19 sections, 6 theorems, 113 equations, 4 figures.

Table of Contents

  1. Introduction
  2. An analysis of data from the Chandra $X$-ray observatory
  3. Modeling framework
  4. Divisible statistics as linear functionals based on the same random measure
  5. Divisible statistics with estimated parameters
  6. Testing background uniformity in sparse $X$-ray spectra via Pearson's $\chi^2$ test
  7. Contiguous alternatives and adequacy for goodness-of-fit
  8. On the behavior of divisible statistics under contiguous alternatives
  9. Distribution of divisible statistics under converging contiguous alternatives
  10. No single divisible statistic is adequate for goodness-of-fit
  11. On the impossibility of testing uniformity via space-homogeneous divisible statistics
  12. Space-homogeneous statistics are unsuitable to test constant background in $X$-ray spectra
  13. Every divisible statistic is dominated by a weighted linear statistic.
  14. Goodness-of-fit via divisible statistics
  15. On the construction of goodness-of-fit tests based on partial sums
  16. ...and 4 more sections

Key Result

Proposition 1

The linear transformation in eqn:projectionB is a projection of $g_{_\theta}\space$ parallel to the function $b_{_\theta}\space$, which defines the estimating equations, and orthogonal to the score function $\psi_{_\theta}\space$, i.e., The projector $\Pi$ is orthogonal when $b_{_\theta}=\psi_{_\theta}$.

Figures (4)

  • Figure 1: $X$-ray source-free spectrum from a Chandra observation and discretized in $350$ bins. The blue solid and red dashed lines are, respectively, the best fits obtained for constant, linear, and piecewise linear mean functions.
  • Figure 2: Mean functions under the null (black solid lines), alternatives (blue dashed lines), and their component detectable by parametric tests obtained by replacing $h$ in \ref{['eqn:mtilde']} with $\widehat{h}$ in \ref{['eqn:hhat']} (red chained lines) for Example I (left panel), Example II (central panel), and Example III (right panel).
  • Figure 3: Left Panel: Comparing the bootstrapped distribution of the Kolmogorov-Smirnov statistics $\widehat{\text{KS}}$ and $\mkern 1.5mu\overline{\mkern-1.5mu\text{KS}\mkern-1.5mu}\mkern 1.5mu$, as defined in \ref{['eqn:KShat']} and \ref{['eqn:KStilde']}, using $100,000$ replicates. Right panel: A realization of the limiting process of $v_{_{\theta\space,\space K}}\space (U_p\Pi_r \ell_{t}\space)$ given by a succession of rescaled independent standard Brownian bridges, defined as in \ref{['bridges']}, when $p=4$.
  • Figure 4: Simulated null distributions of the test statistic $\mkern 1.5mu\overline{\mkern-1.5mu\text{KS}\mkern-1.5mu}\mkern 1.5mu$ in \ref{['eqn:KStilde']} (top panels) and its transformed counterpart $\text{KS}^*$ in \ref{['eqn:KS_star']} shifted by $\frac{0.6}{\sqrt{K}}$ (bottom panels), for the exponential (orange dashed lines) and the normal (blue chained lines) models for $K=50,100,1000$. For comparison, the limiting distribution in \ref{['eqn:kolmogorov']} is also plotted in both panels (black solid lines). Each simulation has been conducted using $100,000$ replicates.

Theorems & Definitions (13)

  • Proposition 1
  • Proposition 2
  • proof
  • Definition 1
  • Proposition 3
  • proof
  • Proposition 4
  • proof
  • Definition 2
  • Proposition 5
  • ...and 3 more