Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Semiparametric KSD test: unifying score and distance-based approaches for goodness-of-fit testing

Zhihan Huang, Ziang Niu

TL;DR

This work reframes GoF testing through exponentially tilted models, showing that score-based tests are equivalent to IPM-based tests when indexed by a suitable function class. It unifies KS, Wasserstein, MMD, and KSD within a single score-based perspective and introduces SKSD, a computationally efficient, semiparametric GoF test using kernelized Stein functions that accommodates nuisance estimators via bootstrap. The SKSD framework is shown to be universally consistent and Pitman efficient, and it applies to models with intractable likelihoods by relying on tractable scores and Stein identities. Through theoretical guarantees and extensive experiments (normality, kernel exponential families, and conditional Gaussian graphs), SKSD demonstrates competitive power with task-specific tests and favorable scalability, suggesting broad applicability for complex GoF diagnostics in high dimensions.

Abstract

Goodness-of-fit (GoF) tests are fundamental for assessing model adequacy. Score-based tests are appealing because they require fitting the model only once under the null. However, extending them to powerful nonparametric alternatives is difficult due to the lack of suitable score functions. Through a class of exponentially tilted models, we show that the resulting score-based GoF tests are equivalent to the tests based on integral probability metrics (IPMs) indexed by a function class. When the class is rich, the test is universally consistent. This simple yet insightful perspective enables reinterpretation of classical distance-based testing procedures-including those based on Kolmogorov-Smirnov distance, Wasserstein-1 distance, and maximum mean discrepancy-as arising from score-based constructions. Building on this insight, we propose a new nonparametric score-based GoF test through a special class of IPM induced by kernelized Stein's function class, called semiparametric kernelized Stein discrepancy (SKSD) test. Compared with other nonparametric score-based tests, the SKSD test is computationally efficient and accommodates general nuisance-parameter estimators, supported by a generic parametric bootstrap procedure. The SKSD test is universally consistent and attains Pitman efficiency. Moreover, SKSD test provides simple GoF tests for models with intractable likelihoods but tractable scores with the help of Stein's identity and we use two popular models, kernel exponential family and conditional Gaussian models, to illustrate the power of our method. Our method achieves power comparable to task-specific normality tests such as Anderson-Darling and Lilliefors, despite being designed for general nonparametric alternatives.

Semiparametric KSD test: unifying score and distance-based approaches for goodness-of-fit testing

TL;DR

This work reframes GoF testing through exponentially tilted models, showing that score-based tests are equivalent to IPM-based tests when indexed by a suitable function class. It unifies KS, Wasserstein, MMD, and KSD within a single score-based perspective and introduces SKSD, a computationally efficient, semiparametric GoF test using kernelized Stein functions that accommodates nuisance estimators via bootstrap. The SKSD framework is shown to be universally consistent and Pitman efficient, and it applies to models with intractable likelihoods by relying on tractable scores and Stein identities. Through theoretical guarantees and extensive experiments (normality, kernel exponential families, and conditional Gaussian graphs), SKSD demonstrates competitive power with task-specific tests and favorable scalability, suggesting broad applicability for complex GoF diagnostics in high dimensions.

Abstract

Goodness-of-fit (GoF) tests are fundamental for assessing model adequacy. Score-based tests are appealing because they require fitting the model only once under the null. However, extending them to powerful nonparametric alternatives is difficult due to the lack of suitable score functions. Through a class of exponentially tilted models, we show that the resulting score-based GoF tests are equivalent to the tests based on integral probability metrics (IPMs) indexed by a function class. When the class is rich, the test is universally consistent. This simple yet insightful perspective enables reinterpretation of classical distance-based testing procedures-including those based on Kolmogorov-Smirnov distance, Wasserstein-1 distance, and maximum mean discrepancy-as arising from score-based constructions. Building on this insight, we propose a new nonparametric score-based GoF test through a special class of IPM induced by kernelized Stein's function class, called semiparametric kernelized Stein discrepancy (SKSD) test. Compared with other nonparametric score-based tests, the SKSD test is computationally efficient and accommodates general nuisance-parameter estimators, supported by a generic parametric bootstrap procedure. The SKSD test is universally consistent and attains Pitman efficiency. Moreover, SKSD test provides simple GoF tests for models with intractable likelihoods but tractable scores with the help of Stein's identity and we use two popular models, kernel exponential family and conditional Gaussian models, to illustrate the power of our method. Our method achieves power comparable to task-specific normality tests such as Anderson-Darling and Lilliefors, despite being designed for general nonparametric alternatives.
Paper Structure (108 sections, 30 theorems, 240 equations, 5 figures, 2 tables, 2 algorithms)

This paper contains 108 sections, 30 theorems, 240 equations, 5 figures, 2 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Our contributions
  3. Unification of score-based test and IPM-based test.
  4. A new nonparametric score-based test.
  5. Related work
  6. Organization of the paper and notations
  7. Spaces and Measures.
  8. Vectors, matrices and functions.
  9. Kernels.
  10. Bridging score-based and IPM-based tests
  11. Exponentially tilted model: a revisit on score-based tests
  12. Nonparametric score-based tests via characteristic IPMs
  13. Kolmogorov-Smirnov (K-S) Distance.
  14. Wasserstein-1 (W-1) Distance.
  15. Maximum Mean Discrepancy (MMD).
  16. ...and 93 more sections

Key Result

Theorem 1

Under Assumption assu:uale and the regularity conditions in Appendix sec:regularity_conditions, we have $T_n(X,\hat{\theta}_n)\overset p \rightarrow \mathrm{KSD}^2(\mathcal{L}_{\theta_0},\mathcal{L})$ as $n\rightarrow\infty$.

Figures (5)

  • Figure 1: Power curves of SKSD goodness-of-fit tests for Gaussian models under various null and alternative distributions. The data generating processes across the subfigures: top left: Gaussian distribution; top right: non-centered Student-$t$ distribution; bottom left: mixture of Gaussians; bottom right: non-centered generalized $\chi^2$ distribution. The black dashed line indicates the test level.
  • Figure 2: Power curve of SKSD goodness-of-fit tests for kernel exponential family. The red dashed line indicates the test level.
  • Figure 3: Top: power curves of SKSD goodness-of-fit tests for quadratic interaction graphical model. The left panel demonstrates the power of our proposed test with varying $\varepsilon$. The right panel shows the power of the same test with fixed $\varepsilon$ and varying $d$. Bottom: the orange curves are the ratios between RMSEs of the estimators and the green curves are the ratios of projection distances between estimators onto the null model, both defined as in \ref{['eq:error_metric']}.
  • Figure 4: Power curves of SKSD goodness-of-fit tests under local alternatives with different estimation procedures. The null model is the Gaussian model $\mathcal{N}(\mu, \sigma^2)$ Left panel: multiplicative local alternatives; Right panel: additive local alternatives. The red dash line indicates the test level.
  • Figure 5: Power curves of SKSD Gaussian goodness-of-fit tests with different distance-based matrices. The left panel uses the Student-$t$ distribution with degree of freedom $\nu$, while the right panel uses the generalized $\chi^2$ distribution with power parameter $\alpha$. The red dash line indicates the test level.

Theorems & Definitions (64)

  • Example 1: Testing mean of Gaussian location shift model, student1908probablerao1948large
  • Example 2: Testing exponentiality under Gamma family, moran1951randomhaywood2008distribution
  • Definition 1: Nonparametric score-based test
  • Definition 2: Uniform asymptotic linear estimate
  • Theorem 1: Consistency of $\mathrm{SKSD}$ test statistic
  • Remark 1: On the limit parameter $\theta_0$
  • Theorem 2: Asymptotic null distribution
  • Remark 2: Proof sketch of Theorem \ref{['thm:asymptotic_distribution']}
  • Remark 3: Computation complexity of Algorithm \ref{['alg:semiparametric-KSD']}
  • Theorem 3: Bootstrap consistency
  • ...and 54 more