OEUVRE: OnlinE Unbiased Variance-Reduced loss Estimation

TL;DR

OEUVRE addresses online loss estimation by producing a current-model loss estimate with a variance-reduced, martingale-based approach that updates in $O(1)$ time using evaluations on consecutive time-step models: $L_t = \ell_t(z_t) + (1-\gamma_t)[L_{t-1} - \ell_{t-1}(z_t)]$. By tying $\gamma_t$ to algorithmic stability bounds, the authors obtain $L^2$ consistency, asymptotic normality, and time-uniform concentration results, with the recursive bound $V^{\uparrow}_t$ governing variance. They prove unbiasedness in total expectation, devise an adaptive scheme to estimate unknown constants like $b$ and $\sigma_t$, and demonstrate strong empirical performance across linear, logistic, expert-advice, and neural-network tasks, often matching or exceeding optimally tuned baselines. The work provides practical online loss estimation with theoretical guarantees, enabling reliable monitoring, early stopping, and model selection in streaming settings. Extensions to non-stationary data and adaptive optimizers are identified as directions for future work.

Abstract

Online learning algorithms continually update their models as data arrive, making it essential to accurately estimate the expected loss at the current time step. The prequential method is an effective estimation approach which can be practically deployed in various ways. However, theoretical guarantees have previously been established under strong conditions on the algorithm, and practical algorithms have hyperparameters which require careful tuning. We introduce OEUVRE, an estimator that evaluates each incoming sample on the function learned at the current and previous time steps, recursively updating the loss estimate in constant time and memory. We use algorithmic stability, a property satisfied by many popular online learners, for optimal updates and prove consistency, convergence rates, and concentration bounds for our estimator. We design a method to adaptively tune OEUVRE's hyperparameters and test it across diverse online and stochastic tasks. We observe that OEUVRE matches or outperforms other estimators even when their hyperparameters are tuned with oracle access to ground truth.

Figures (12)

• Figure 1: We illustrate the behavior of OEUVRE and several baselines on a representative run of the Diabetes Health Indicators dataset. The hyperparameter for each baseline was chosen using grid search to minimize RMSE. We see that our proposed estimator provides a more accurate continuous estimate of the true loss compared to the baselines without the need for hyperparameter tuning.
• Figure 2: Performance comparison of OEUVRE against the best baseline and the median baseline for the online linear regression task. OEUVRE achieves competitive RMSE and bias when compared to the best baseline.
• Figure 3: Comparison of OEUVRE with best-performing baseline methods for (a) prediction with expert advice task and (b) neural networks task. In (a), it achieves smaller RMSE and comparable MAE when compared to the best baseline. In (b), it achieves the lowest RMSE and MAE for both datasets.
• Figure 4: Performance of different loss estimation methods on the MNIST and EMNIST dataset with batch size 8.
• Figure 5: Performance comparison of OEUVRE against the best hyperparameter setting for each baseline for the linear regression task. OEUVRE achieves competitive RMSE, MAE, and bias when compared to the best baseline.

Theorems & Definitions (24)

• Proposition 1
• Proposition 2
• Lemma 3
• Theorem 4
• Definition 5
• Proposition 6
• Theorem 7
• Theorem 8
• Lemma 9
• proof