Risk-Averse Learning with Varying Risk Levels

TL;DR

This work tackles risk-averse online optimization in nonstationary environments by optimizing the time-varying CVaR objective $C_t(x)=CVaR_{\alpha_t}[J_t(x,\xi)]$, where both the cost function and the risk level can change over time. It introduces two variation metrics, function variation $V_f$ and risk-level variation $V_\alpha$, and develops both first-order and zeroth-order algorithms that operate under a finite sampling budget. Theoretical results provide dynamic regret bounds that scale with $V_f$, $V_\alpha$, and the total number of samples, demonstrating how adaptation to environmental and risk fluctuations yields sublinear regret when cumulative variation grows sublinearly. Numerical experiments in dynamic pricing illustrate faster convergence and lower CVaR and regret for the gradient-based method, validating the advantage of using first-order information in risk-averse online optimization with varying risk and nonstationarity.

Abstract

In safety-critical decision-making, the environment may evolve over time, and the learner adjusts its risk level accordingly. This work investigates risk-averse online optimization in dynamic environments with varying risk levels, employing Conditional Value-at-Risk (CVaR) as the risk measure. To capture the dynamics of the environment and risk levels, we employ the function variation metric and introduce a novel risk-level variation metric. Two information settings are considered: a first-order scenario, where the learner observes both function values and their gradients; and a zeroth-order scenario, where only function evaluations are available. For both cases, we develop risk-averse learning algorithms with a limited sampling budget and analyze their dynamic regret bounds in terms of function variation, risk-level variation, and the total number of samples. The regret analysis demonstrates the adaptability of the algorithms in non-stationary and risk-sensitive settings. Finally, numerical experiments are presented to demonstrate the efficacy of the methods.

Figures (7)

• Figure 1: Top: The parking prices generated by the first-order learning algorithm (Algorithm \ref{['alg:first-order']}), the zeroth-order learning algorithm (Algorithm \ref{['alg:zeroth-order']}), and the brute-force optimal price $x_t^\ast$, under the desired occupancy \ref{['eq:desired occupancy step']} and risk level \ref{['eq:risk level step']}. Bottom: The resulting occupancies for each price trajectory.
• Figure 2: Top: The CVaR values under the price generated by the first-order algorithm (Algorithm \ref{['alg:first-order']}), by the zeroth-order algorithm (Algorithm \ref{['alg:zeroth-order']}), and the brutal-force optimal price $x_t^\ast$, under the desired occupancy \ref{['eq:desired occupancy step']} and the risk level \ref{['eq:risk level step']}. Bottom: The corresponding dynamic regret.
• Figure 3: From top to bottom: the parking prices generated by the first-order algorithm (Algorithm \ref{['alg:first-order']}), the zeroth-order algorithm (Algorithm \ref{['alg:zeroth-order']}), and the brute-force optimal price $x_t^\ast$, under the target occupancy $r_t^d = 0.7 + 0.05\cos(2\pi t/500)$ and the risk level $\alpha_t = 0.5 + 0.3\cos(2\pi t/500)$; the corresponding occupancies under each price trajectory; and the resulting dynamic regret.
• Figure 4: From top to bottom: the prices generated by Algorithm \ref{['alg:first-order']} and the brutal-force optimal prices $x_t^\ast$, under the function variations $V_f^1$, $V_f^2$ and $V_f^3$, respectively; and the resulting dynamic regret.
• Figure 5: From top to bottom: the prices generated by Algorithm \ref{['alg:first-order']} and the brutal-force optimal prices $x_t^\ast$, under the risk-level variations $V_\alpha^1$, $V_\alpha^2$ and $V_\alpha^3$, respectively; and the resulting dynamic regret.

Theorems & Definitions (16)

• Definition 1
• Lemma 1
• Definition 2
• Lemma 2
• Lemma 3
• Lemma 4
• Lemma 5
• Theorem 1
• Remark 1
• Remark 2