Model-free Bootstrap and Conformal Prediction in Regression: Conditionality, Conjecture Testing, and Pertinent Prediction Intervals
Yiren Wang, Dimitris N. Politis
TL;DR
This paper analyzes predictive inference in regression for constructing prediction intervals under three frameworks: naive quantile estimation, conformal prediction, and Model-free Bootstrap (MFB). It emphasizes conditional validity, shows asymptotic conditional coverage holds for QE, CP, and MFB, and introduces the notion of asymptotic pertinence to capture estimation variability—showing MFB can yield better finite-sample conditional coverage than QE. It also develops conjecture testing as an analogue to hypothesis testing for future responses and demonstrates one-sided conjecture testing with MFB and CP adaptations, supported by extensive simulations and a VaR-related financial example. Together, the work provides guidance on when to use each method, highlights the robustness of MFB under conditioning, and contributes new theoretical concepts to improve predictive interval performance in finite samples and under conditioning.
Abstract
Predictive inference under a general regression setting is gaining more interest in the big-data era. In terms of going beyond point prediction to develop prediction intervals, two main threads of development are conformal prediction and Model-free prediction. Recently, a new conformal prediction approach was proposed that exploits the same uniformization procedure as in the well-known Model-free Bootstrap. Hence, it is of interest to compare and further investigate the performance of the two methods. In the paper at hand, we contrast the two approaches via theoretical analysis and numerical experiments with a focus on conditional coverage of prediction intervals. We discuss suitable scenarios for applying each algorithm, underscore the importance of conditional vs. unconditional coverage, and show that, under mild conditions, the Model-free bootstrap yields prediction intervals with guaranteed better conditional coverage compared to quantile estimation. We also extend the concept of 'pertinence' of prediction intervals to the nonparametric regression setting, and give concrete examples where its importance emerges under finite sample scenarios. Finally, we define the new notion of 'conjecture testing' that is the analog of hypothesis testing as applied to the prediction problem; we also devise a modified conformal score to allow conformal prediction to handle one-sided 'conjecture tests', and compare to the Model-free bootstrap.