Simultaneous Swap Regret Minimization via KL-Calibration
Haipeng Luo, Spandan Senapati, Vatsal Sharan
TL;DR
This work introduces (pseudo) KL-Calibration as a stronger, KL-divergence-based notion of online calibration for sequential binary predictions, linking calibration quality to swap regret, particularly for the log loss. It shows that KL-Calibration bounds swap regret for bounded proper losses with twice differentiable univariate forms (L_2) and, for G-smooth losses (L_G), implies near-optimal sublinear swap regrets, including high-probability guarantees. The authors establish both non-constructive existence results via minimax duality and an explicit Blum–Mansour-based algorithm with a non-uniform discretization and randomized rounding, achieving O(T^{1/3}) pseudo KL-Cal and O(T^{1/3} (log T)^{2/3}) PKLCal, respectively, for broad loss families. A key technical contribution is using Freedman-type martingale bounds and a carefully designed discretization to obtain sublinear calibration and swap-regret bounds, with Tsallis-type and log losses included in the analysis. The results advance online calibration, enabling robust decision-making across multiple losses and offering new directions for high-probability calibration in the offline/online settings.”
Abstract
Calibration is a fundamental concept that aims at ensuring the reliability of probabilistic predictions by aligning them with real-world outcomes. There is a surge of studies on new calibration measures that are easier to optimize compared to the classical $\ell_1$-Calibration while still having strong implications for downstream applications. One recent such example is the work by Fishelson et al. (2025) who show that it is possible to achieve $O(T^{1/3})$ pseudo $\ell_2$-Calibration error via minimizing pseudo swap regret of the squared loss, which in fact implies the same bound for all bounded proper losses with a smooth univariate form. In this work, we significantly generalize their result in the following ways: (a) in addition to smooth univariate forms, our algorithm also simultaneously achieves $O(T^{1/3})$ swap regret for any proper loss with a twice continuously differentiable univariate form (such as Tsallis entropy); (b) our bounds hold not only for pseudo swap regret that measures losses using the forecaster's distributions on predictions, but also hold for the actual swap regret that measures losses using the forecaster's actual realized predictions. We achieve so by introducing a new stronger notion of calibration called (pseudo) KL-Calibration, which we show is equivalent to the (pseudo) swap regret for log loss. We prove that there exists an algorithm that achieves $O(T^{1/3})$ KL-Calibration error and provide an explicit algorithm that achieves $O(T^{1/3})$ pseudo KL-Calibration error. Moreover, we show that the same algorithm achieves $O(T^{1/3}(\log T)^{-1/3}\log(T/δ))$ swap regret w.p. $\ge 1-δ$ for any proper loss with a smooth univariate form, which implies $O(T^{1/3})$ $\ell_2$-Calibration error. A technical contribution of our work is a new randomized rounding procedure and a non-uniform discretization scheme to minimize the swap regret for log loss.