In Defense of Defensive Forecasting
Juan Carlos Perdomo, Benjamin Recht
TL;DR
This work surveys Defensive Forecasting, a game-theoretic framework that designs predictions to correct past mistakes against adversarial outcomes, rather than forecasting the future. It introduces a unifying meta-algorithm based on a fundamental inequality and anticorrelation searches, and then instantiates it across online learning, calibration, expert prediction, and online conformal-style tasks, including kernelized and RKHS extensions. The authors derive near-optimal regret bounds for squared and log losses, provide calibration guarantees (including smooth calibration via the Fermi–Sobolev kernel), and present online-to-batch reductions that give strong batch guarantees. The approach yields practical, adaptive algorithms with theoretical guarantees in both finite and infinite-dimensional feature spaces, and connects to outcome indistinguishability and conformal ideas, offering a cohesive toolkit for robust sequential prediction in adversarial and stochastic settings.
Abstract
This tutorial provides a survey of algorithms for Defensive Forecasting, where predictions are derived not by prognostication but by correcting past mistakes. Pioneered by Vovk, Defensive Forecasting frames the goal of prediction as a sequential game, and derives predictions to minimize metrics no matter what outcomes occur. We present an elementary introduction to this general theory and derive simple, near-optimal algorithms for online learning, calibration, prediction with expert advice, and online conformal prediction.