Allocating Variance to Maximize Expectation

TL;DR

This work studies how to allocate a fixed variance budget among Gaussian variables to maximize the expected aggregate performance, either a single-set maximum $\mathbb{E}[\max_i X_i]$ or a sum over multiple sets $\mathbb{E}[\sum_{j=1}^m \max_{i\in S_j} X_i]$. It develops PTASes for the independent and correlated single-set problems by exploiting a structural bound that most variance can be ignored below a small threshold and by grid-discretizing the remaining decision space; for the multi-set case it yields a provable $\Omega\left(\frac{1}{\log n}\right)$-approximation via a bucketed, submodular-maximization approach. The analysis leverages a key eps-contribution lemma, Lipschitz-type smoothness, and Earth Mover’s Distance arguments to relate nearby distributions, enabling efficient approximation despite non-convex objectives and correlation. The results have implications for variance shaping in ML reliability, mixture-model learning, and revenue optimization in auctions, where tuning prediction uncertainty directly impacts performance.

Abstract

We design efficient approximation algorithms for maximizing the expectation of the supremum of families of Gaussian random variables. In particular, let $\mathrm{OPT}:=\max_{σ_1,\cdots,σ_n}\mathbb{E}\left[\sum_{j=1}^{m}\max_{i\in S_j} X_i\right]$, where $X_i$ are Gaussian, $S_j\subset[n]$ and $\sum_iσ_i^2=1$, then our theoretical results include: - We characterize the optimal variance allocation -- it concentrates on a small subset of variables as $|S_j|$ increases, - A polynomial time approximation scheme (PTAS) for computing $\mathrm{OPT}$ when $m=1$, and - An $O(\log n)$ approximation algorithm for computing $\mathrm{OPT}$ for general $m>1$. Such expectation maximization problems occur in diverse applications, ranging from utility maximization in auctions markets to learning mixture models in quantitative genetics.

Figures (2)

• Figure 1: Concavity of optimal objective value per set (i.e. $\frac{1}{m}\textsc{OPT}$) when the Gaussians are restricted to be independent, be positively, or be negatively correlated in $\mathsf{GraphVarAlloc}$, as a function of $p$ for Erdós-Renyi graphs with $p\in\{\frac{1}{8},\frac{2}{8},...,\frac{8}{8}\}$ and $n=8$. To improve the simulation speed the positive and negative correlations are limited to block-diagonal matrices with block sizes $2\times2$, and $\Sigma_{2i+1,2i+2}=\pm\sqrt{\Sigma_{2i+1,2i+1}\cdot\Sigma_{2i+2,2i+2}}$. See also Theorem \ref{['thm:per-set-submodular']} for the discussion of the independent case.
• Figure 2: Simulations for Erdós-Renyi graphs $G(n,p)$ for $\mathsf{GraphVarAlloc}$ ($n$ is held constant). $x$-axis corresponds to the $n$ variances (sorted by increasing values) and $y$-axis corresponds to their optimal values in allocation. Note the increasingly concentrated variance allocation with increasing $p$. See also Theorem \ref{['thm:random-graph-ordering']}.

Theorems & Definitions (48)

• Theorem 1.1
• Theorem 1.2
• Theorem 1.3
• Theorem 1.4
• Theorem 1.5
• Theorem 1.6
• Lemma 2.0
• proof : Proof of Lemma \ref{['lem:eps-contribution']}
• Lemma 2.0
• Lemma 2.0