Bootstrap Diagnostic Tests

TL;DR

This paper develops bootstrap-based diagnostics to detect violations of Gaussian limit theory and false specifications that undermine standard inference. By comparing the conditional bootstrap distribution of a statistic to the Gaussian limit under valid specification, and employing a joint $(n,m)$ asymptotic regime, it yields tests that are size-controlled and free of pre-testing bias for post-diagnostic inference. The key idea is that, under valid specification, the bootstrap replicates a Gaussian limit, while under invalid specification the bootstrap distribution is random and non-Gaussian, enabling consistent detection across settings such as weak instruments, non-stationarity, boundary parameters, infinite variance, and near-singular Jacobians. The approach is computationally straightforward, applicable with a common set of critical values, and is illustrated with an empirical macroeconomic application showing its practical relevance and broad applicability. Overall, the bootstrap diagnostics offer a flexible, theoretically grounded tool for validating specification and safeguarding subsequent inference.

Abstract

Violation of the assumptions underlying classical (Gaussian) limit theory often yields unreliable statistical inference. This paper shows that the bootstrap can detect such violations by delivering simple and powerful diagnostic tests that (a) induce no pre-testing bias, (b) use the same critical values across applications, and (c) are consistent against deviations from asymptotic normality. The tests compare the conditional distribution of a bootstrap statistic with the Gaussian limit implied by valid specification and assess whether the resulting discrepancy is large enough to indicate failure of the asymptotic Gaussian approximation. The method is computationally straightforward and only requires a sample of i.i.d. draws of the bootstrap statistic. We derive sufficient conditions for the randomness in the data to mix with the randomness in the bootstrap repetitions in a way such that (a), (b) and (c) above hold. We demonstrate the practical relevance and broad applicability of bootstrap diagnostics by considering several scenarios where the asymptotic Gaussian approximation may fail, including weak instruments, non-stationarity, parameters on the boundary of the parameter space, infinite variance data and singular Jacobian in applications of the delta method. An illustration drawn from the empirical macroeconomic literature concludes.

Figures (2)

• Figure 1: Fan chart of $M=1{,}000$ i.i.d. realizations of the (conditional) cdf $\hat{G}_{n}$ of $T_{n}^{\ast}$ under weak and strong instruments ($n = 1{,}000$).
• Figure 2: Bootstrap diagnostics. Upper panel: bootstrap (conditional) cdfs for the on-impact IRFs (solid lines) against the Gaussian distribution (dashed lines). Lower panel: average fractions of rejections ($\hat{\pi}_{n,m,K}^{\ast}(\eta)$) for nominal levels $\eta\in(0,0.10)$, $m=10$ (solid lines) and $m=20$ (dashed lines).