Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On the Precise Asymptotics of Universal Inference

Kenta Takatsu

TL;DR

This work precisely characterizes why universal inference can be excessively conservative under model misspecification and regularity conditions, revealing a fundamental gap between validity and practical conservativeness. It introduces a principled remedy based on studentization and bias correction, yielding an asymptotically exact $1-\alpha$ coverage when the product of estimation errors is negligible, a property that resembles double robustness in semiparametric theory. The results rely on a quantitative Berry-Esseen refinement and sample-splitting to decouple estimators, and they extend to a broader class of e-value procedures beyond universal inference. The findings have direct implications for high-dimensional inference, clarifying dimension-dependent limits and offering a practical pathway to tighter, reliable confidence sets in misspecified models.

Abstract

In statistical inference, confidence set procedures are typically evaluated based on their validity and width properties. Even when procedures achieve rate-optimal widths, confidence sets can still be excessively wide in practice due to elusive constants, leading to extreme conservativeness, where the empirical coverage probability of nominal $1-α$ level confidence sets approaches one. This manuscript studies this gap between validity and conservativeness, using universal inference (Wasserman et al., 2020) with a regular parametric model under model misspecification as a running example. We identify the source of asymptotic conservativeness and propose a general remedy based on studentization and bias correction. The resulting method attains exact asymptotic coverage at the nominal $1-α$ level, even under model misspecification, provided that the product of the estimation errors of two unknowns is negligible, exhibiting an intriguing resemblance to double robustness in semiparametric theory.

On the Precise Asymptotics of Universal Inference

TL;DR

This work precisely characterizes why universal inference can be excessively conservative under model misspecification and regularity conditions, revealing a fundamental gap between validity and practical conservativeness. It introduces a principled remedy based on studentization and bias correction, yielding an asymptotically exact coverage when the product of estimation errors is negligible, a property that resembles double robustness in semiparametric theory. The results rely on a quantitative Berry-Esseen refinement and sample-splitting to decouple estimators, and they extend to a broader class of e-value procedures beyond universal inference. The findings have direct implications for high-dimensional inference, clarifying dimension-dependent limits and offering a practical pathway to tighter, reliable confidence sets in misspecified models.

Abstract

In statistical inference, confidence set procedures are typically evaluated based on their validity and width properties. Even when procedures achieve rate-optimal widths, confidence sets can still be excessively wide in practice due to elusive constants, leading to extreme conservativeness, where the empirical coverage probability of nominal level confidence sets approaches one. This manuscript studies this gap between validity and conservativeness, using universal inference (Wasserman et al., 2020) with a regular parametric model under model misspecification as a running example. We identify the source of asymptotic conservativeness and propose a general remedy based on studentization and bias correction. The resulting method attains exact asymptotic coverage at the nominal level, even under model misspecification, provided that the product of the estimation errors of two unknowns is negligible, exhibiting an intriguing resemblance to double robustness in semiparametric theory.

Paper Structure

This paper contains 19 sections, 10 theorems, 137 equations, 4 figures, 1 table.

Table of Contents

  1. Introduction
  2. Definitions and Main Claims
  3. Main claims
  4. Scope of this manuscript
  5. Notation
  6. Maximimum Likelihood under Misspecification
  7. Extreme Conservativeness of Universal Inference
  8. Studentized and Bias-corrected Universal Inference
  9. Illustration --- Misspecified Linear Regression
  10. Summary and Conclusions
  11. Acknowledgments
  12. Proof of Theorem \ref{['thm:ui-conservatism']}
  13. Proofs of Corollaries \ref{['cor:trivial-conservative']} and \ref{['cor:trivial-correct']}
  14. Asymptotic Results in Fixed Dimension
  15. Proof of Theorem \ref{['thm:bc-conservatism']}
  16. ...and 4 more sections

Key Result

Theorem 1

Assume as:QMD, as:hessian and as:holder_inequality. Define the Kolmogorov-Smirnov distance for $G \in \mathcal{G}$ as where $\xi = \log p_{\widehat{\theta}_1}(Z) - \log p_{\theta_G}(Z)$ and $\sigma^2_{\theta_G, \widehat{\theta}_1} = \mathrm{Var}_{G}[\xi | D_1]$. The nonasymptotic miscoverage probability of the universal confidence set for any $G$ satisfies where and with a constant $C_p$ only

Figures (4)

  • Figure 1: A schematic illustrating the sample splitting procedure. The arrow indicates that the objects are derived from the corresponding data. The left panel shows the proposed procedure, while the right panel provides the illustration used in Remark \ref{['remark:sample-splitting']}.
  • Figure 2: Comparison of the empirical coverage of $95\%$ confidence sets for fixed dimension with varying sample size (left panel) and fixed sample size with varying dimension (right panel). The empirical coverage is computed from 1000 replications. The methods are labeled as: Universal Inference (Correct) based on \ref{['eq:ui-def-model']}, Universal Inference (Incorrect) based on \ref{['eq:ui-def-incorrect']}, Studentized based on \ref{['eq:std-ui-lr']}, and Studentized + Bias-corrected based on \ref{['eq:bc-ui-lr']}. The left panel shows that as the sample size increases, both Universal Inference (Correct) and Studentized are conservative, Universal Inference (Incorrect) is invalid, and Studentized + Bias-corrected achieves nominal coverage. The right panel shows that as the dimension increases, Universal Inference (Correct) and Studentized are extremely conservative with the coverage probability of $1$, Universal Inference (Incorrect) transitions from invalid to extremely conservative, and Studentized + Bias-corrected transitions from nominal to extremely conservative. These findings align with the theoretical expectations discussed in the manuscript.
  • Figure A.1: A schematic illustrating the sample splitting procedure. The arrow indicates that the objects are derived from the corresponding data. The left panel shows the proposed procedure, while the right panel provides the illustration used in this section
  • Figure A.2: Comparison of the empirical coverage of $95\%$ confidence sets for fixed dimension with varying sample size (left panel) and fixed sample size with varying dimension (right panel). The empirical coverage is computed from 1000 replications. The methods are labeled as: Universal Inference 1 based on \ref{['eq:ui-def-model']}, Universal Inference 2 based on \ref{['eq:ui-def-incorrect']}, Studentized based on \ref{['eq:std-ui-lr']}, and Studentized + Bias-corrected based on \ref{['eq:bc-ui-lr']}. It is incorrect to interpret that Universal Inference 1 performs better, as its coverage now approaches the nominal level. This merely reflects the fact that the validity of universal inference under model misspecification can be arbitrary, ranging from conservative, exact to invalid. While conservative, the validity of the Studentized method remains robust under model misspecification, demonstrating the benefit of studentization even when bias correction is infeasible.

Theorems & Definitions (27)

  • Definition 1
  • Definition 2
  • Definition 3
  • Definition 4: QMD under Misspecification
  • Definition 5: Uniform Lindeberg condition
  • Theorem 1
  • Corollary 1.1
  • Corollary 1.2
  • Theorem 2
  • Remark 1: On the estimation of $V_{G}$
  • ...and 17 more