Locally Adaptive Multi-Objective Learning

TL;DR

This work addresses online learning under arbitrary distribution shifts when multiple objectives must be satisfied simultaneously. It introduces a locally adaptive framework that replaces the standard Hedge weight-update with a Fixed Share-based online learner to guarantee interval-wise (local) performance, yielding time-local guarantees for any interval $I$ with losses bounded by $O\left(\sqrt{\frac{\log(|\mathcal{L}|\cdot |I|)}{|I|}}\right)$. The approach is instantiated to multiaccuracy mean estimation and quantile estimation, augmented by a prediction-error objective to preserve baseline predictive accuracy, and extended to online quantile loss via pinball loss. Empirically, on GEFCom2014-L and COMPAS datasets, locally adaptive MA+pred achieves near-zero local multiaccuracy errors and maintains or improves predictive accuracy relative to baselines, outperforming adaptive-objectives variants in local performance. The results suggest that time-local guarantees and adaptivity can significantly improve fairness and calibration under distribution shifts in real-world sequential prediction tasks, with code released for reproducibility.

Abstract

We consider the general problem of learning a predictor that satisfies multiple objectives of interest simultaneously, a broad framework that captures a range of specific learning goals including calibration, regret, and multiaccuracy. We work in an online setting where the data distribution can change arbitrarily over time. Existing approaches to this problem aim to minimize the set of objectives over the entire time horizon in a worst-case sense, and in practice they do not necessarily adapt to distribution shifts. Earlier work has aimed to alleviate this problem by incorporating additional objectives that target local guarantees over contiguous subintervals. Empirical evaluation of these proposals is, however, scarce. In this article, we consider an alternative procedure that achieves local adaptivity by replacing one part of the multi-objective learning method with an adaptive online algorithm. Empirical evaluations on datasets from energy forecasting and algorithmic fairness show that our proposed method improves upon existing approaches and achieves unbiased predictions over subgroups, while remaining robust under distribution shift.

Figures (16)

• Figure 1: GEFCom14-L electric load forecasting dataset. On the left hand side are the time series for the raw load (light brown) and temperature (light orange) data. The dark brown curves indicate the weekly (168-hourly) moving average. The shaded grey region shows the competition duration. On the right-hand side, we plot a weekly moving average of the local multiaccuracy error.
• Figure 2: COMPAS dataset. Moving average of true recidivism over time. We show 30-day moving averages of $y$ (recidivism indicator), computed overall and separately by racial group. For each calendar date, outcomes are first averaged across all individuals screened that day and then reported as a 30-day time-window rolling mean.
• Figure 3: Local multiaccuracy error (left) and prediction error (right) on the GEFCom2014-L dataset. We skip the first ten time steps when plotting the multiaccuracy and prediction error for improved readability.
• Figure 4: Local multiaccuracy error (left) and prediction error (right) on the COMPAS dataset. We skip the first two time steps when plotting the multiaccuracy error for improved readability.
• Figure 5: Local multiaccuracy error on GEFCom2014-L for different interval widths. We skip the first thirty time steps when plotting the multiaccuracy error for improved readability.

Theorems & Definitions (9)

• Definition 1: Online multiaccuracy
• Definition 2: Online multicalibration
• Definition 3: Online prediction error
• Theorem 2.1: Theorem 2.1 in lee2022online
• Lemma 3.1
• Lemma 3.2
• Theorem 3.1
• Corollary 4.1
• Corollary 4.2