Bayes Risk for Goodness of Fit Tests

TL;DR

This work reframes GOF testing under a Bayesian decision-theoretic lens and demonstrates that Bayes-risk optimization drives GOF calibrations to a moderate-deviation scale, yielding thresholds that inflate like $t_n^*\sim\sqrt{\frac{\kappa}{4\rho}\log n}$ and Type I errors that decay polynomially. It unifies KS-type empirical-process GOF, Sanov-based tests, and parametric GOF through a two-term risk decomposition governed by the tail parameter $\rho$ and the local prior mass exponent $\kappa$, connecting to Sanov information asymptotics and Fisher information geometry. The framework is instantiated across multiple GOF settings—location testing under Laplace models, shape testing via Bayes factors, multinomial GOF, and contingency-table independence tests—with numerical verification confirming the MDp calibration. The results offer principled, sample-size-dependent calibration guidelines and illuminate practical implications for statistical practice in scientific and regulatory contexts.

Abstract

We develop a unified framework for goodness-of-fit (GOF) testing through the lens of Bayes risk. Classical GOF procedures are commonly calibrated either at fixed significance level (CLT scale) or through exponential error exponents (LDP scale). We establish that Bayes-risk optimal calibration operates on the moderate-deviation (MDP) scale, producing canonical $\sqrt{\log n}$ inflation of rejection thresholds and polynomially decaying Type I error. Our main contributions are: (i) we formalise the Rubin--Sethuraman program for KS-type statistics as a risk-calibration theorem with explicit regularity conditions on priors and empirical-process functionals; (ii) we develop the precise connection between Bayes-risk expansions and Sanov information asymptotics, showing how $\log n$-order truncations arise naturally when risk, rather than pure exponents, is the evaluation criterion; (iii) we provide detailed applications to location testing under Laplace families, shape testing via Bayes factors, and connections to Fisher information geometry. The organizing principle throughout is that sample size enters Bayes-optimal GOF cutoffs through the MDP scale, unifying KS-based and Sanov-based perspectives under a single risk criterion.

Figures (7)

• Figure 1: The three calibration regimes. Only the moderate-deviation (MDP) scale achieves vanishing Bayes risk at the optimal rate.
• Figure 2: Risk decomposition geometry. The Type I term (solid, decreasing) and Type II term (dashed, increasing) cross at the unique minimiser $a^{*}=\kappa/(4\rho)$. The total risk $B_n(a)$ (grey) achieves its minimum at the MDP scale.
• Figure 3: KL truncation geometry. LDP optimises the full exponent (dashed); Bayes risk truncates to the $O(\log n/n)$ shell (dotted), where prior mass and detectability compete. The shaded region is the MDP-active zone.
• Figure 4: Fisher geometry of the MDP boundary. The prior $\Pi_1$ places mass $\asymp\varepsilon^{\kappa}$ in the critical shell of Fisher radius $\sqrt{\log n/n}$ around the null $\theta_0$. The volume element contributes $\varepsilon^{d}$ (dimension) and the density contributes $\varepsilon^{\lambda}$ (prior exponent).
• Figure 5: Bayes risk $B_n(a)$ vs. threshold parameter $a$ (where $t_n=\sqrt{a\log n}$) for the KS statistic with $\rho=1$ and $\kappa=2$. Dots: numeric minimiser at each $n$. Dashed line: asymptotic optimum $a^{\star}=0.5$. The minimiser converges to $a^{\star}$ as $n\to\infty$.

Theorems & Definitions (42)

• Definition 2.1: Bayes risk for testing
• Remark 2.2: Error probabilities and averaged errors
• Proposition 2.3: Bayes-optimal test
• proof
• Definition 2.4: Calibration regimes
• Theorem 2.6: MDP is Bayes-risk optimal for GOF
• proof
• Remark 2.7
• Lemma 2.8: Risk Decomposition Template
• proof