Multi-Objective $\textit{min-max}$ Online Convex Optimization

TL;DR

This work extends online convex optimization to a multi-objective framework via a min-max regret criterion over $K$ loss sequences in an iid setting. It introduces a simple Hedge plus online gradient descent algorithm that alternates learning a weight over objectives with optimizing a surrogate objective, achieving an expected min-max static regret of $O(sqrt(T))$ for $K=2$ and $O(sqrt(T log K))$ for general $K$, with extensions to bandit feedback. The paper also strengthens results for the GlobalExpertsProblem under iid inputs and provides empirical evidence supporting the theoretical bounds, including applications to fair classification. It distinguishes its min-max formulation from COCO and global-experts benchmarks and outlines several open questions, including worst-case guarantees and constrained variants.

Abstract

In online convex optimization (OCO), a single loss function sequence is revealed over a time horizon of $T$, and an online algorithm has to choose its action at time $t$, before the loss function at time $t$ is revealed. The goal of the online algorithm is to incur minimal penalty (called $\textit{regret}$ compared to a static optimal action made by an optimal offline algorithm knowing all functions of the sequence in advance. In this paper, we broaden the horizon of OCO, and consider multi-objective OCO, where there are $K$ distinct loss function sequences, and an algorithm has to choose its action at time $t$, before the $K$ loss functions at time $t$ are revealed. To capture the tradeoff between tracking the $K$ different sequences, we consider the $\textit{min-max}$ regret, where the benchmark (optimal offline algorithm) takes a static action across all time slots that minimizes the maximum of the total loss (summed across time slots) incurred by each of the $K$ sequences. An online algorithm is allowed to change its action across time slots, and its {\it min-max} regret is defined as the difference between its $\textit{min-max}$ cost and that of the benchmark. The $\textit{min-max}$ regret is a stringent performance measure and an algorithm with small regret needs to `track' all loss function sequences closely at all times. We consider this $\textit{min-max}$ regret in the i.i.d. input setting where all loss functions are i.i.d. generated from an unknown distribution. For the i.i.d. model we propose a simple algorithm that combines the well-known $\textit{Hedge}$ and online gradient descent (OGD) and show via a remarkably simple proof that its expected $\textit{min-max}$ regret is $O(\sqrt{T \log K})$.

Figures (5)

• Figure 1: Expected min-max static regret of Algorithm \ref{['alg:main']} when $f_t^k$ are random linear functions for different values of $K$.
• Figure 2: Expected min-max static regret of Algorithm \ref{['alg:main']} when $f_t^k$ are random quadratic functions for different values of $K$.
• Figure 3: Expected min-max static regret of MULTIMannorGlobal and Algorithm \ref{['alg:main']} for different numbers of experts $K$ and time horizons $T$ for the fair classification example.
• Figure 4: Expected min-max static regret comparison for $K=10$.
• Figure 5: Expected min-max static regret for Algorithm \ref{['alg:main']} vs. FTRL in a piecewise-stationary environment.

Theorems & Definitions (24)

• Remark 1: On the regret definition
• Remark 2
• Remark 3
• Remark 4
• Theorem 4
• proof
• Lemma 5
• Lemma 6
• Lemma 7
• Remark 5