Dimension-Free Decision Calibration for Nonlinear Loss Functions
Jingwu Tang, Jiayun Wu, Zhiwei Steven Wu, Jiahao Zhang
TL;DR
The paper tackles dimension-free decision calibration for nonlinear loss functions in high-dimensional outcome spaces, where full calibration is computationally prohibitive. It first demonstrates a lower bound of $\Omega(\sqrt{m})$ samples for verifying decision calibration under deterministic optimal decision rules, motivating a shift to a smooth decision rule (quantal response) that yields dimension-free auditing. It then develops the DimFreeDeCal algorithm that post-processes any predictor using $\mathrm{poly}(|\mathcal{A}|,1/\epsilon)$ samples to achieve $(\mathcal{L}_\mathcal{H},\tilde{\mathcal{K}}_{\mathcal{L}_\mathcal{H}},\epsilon)$-decision calibration, with kernel-based, RKHS-enabled computations that are oracle-efficient given an auditing subroutine. The framework handles loss classes that admit uniform $\dim(\mathcal{H})$-bounded approximations via $\phi(y)$ and applies to infinite-dimensional feature mappings, enabling dimension-free guarantees for nonlinear losses through weighted calibration and implicit patching in RKHS. Overall, the work provides principled, scalable tools to ensure decision-makers can reliably respond to predictions without requiring dimension-dependent samples, with practical implications for high-stakes domains where outcome spaces are large or continuous.
Abstract
When model predictions inform downstream decision making, a natural question is under what conditions can the decision-makers simply respond to the predictions as if they were the true outcomes. Calibration suffices to guarantee that simple best-response to predictions is optimal. However, calibration for high-dimensional prediction outcome spaces requires exponential computational and statistical complexity. The recent relaxation known as decision calibration ensures the optimality of the simple best-response rule while requiring only polynomial sample complexity in the dimension of outcomes. However, known results on calibration and decision calibration crucially rely on linear loss functions for establishing best-response optimality. A natural approach to handle nonlinear losses is to map outcomes $y$ into a feature space $φ(y)$ of dimension $m$, then approximate losses with linear functions of $φ(y)$. Unfortunately, even simple classes of nonlinear functions can demand exponentially large or infinite feature dimensions $m$. A key open problem is whether it is possible to achieve decision calibration with sample complexity independent of~$m$. We begin with a negative result: even verifying decision calibration under standard deterministic best response inherently requires sample complexity polynomial in~$m$. Motivated by this lower bound, we investigate a smooth version of decision calibration in which decision-makers follow a smooth best-response. This smooth relaxation enables dimension-free decision calibration algorithms. We introduce algorithms that, given $\mathrm{poly}(|A|,1/ε)$ samples and any initial predictor~$p$, can efficiently post-process it to satisfy decision calibration without worsening accuracy. Our algorithms apply broadly to function classes that can be well-approximated by bounded-norm functions in (possibly infinite-dimensional) separable RKHS.