Numerical solution of a PDE arising from prediction with expert advice

TL;DR

The paper links online prediction with expert advice in an adversarial setting to a degenerate elliptic PDE that governs asymptotically optimal strategies via its viscosity solution. By exploiting permutation symmetry and a translation-based dimension reduction, the authors devise a robust finite-difference framework and domain reductions to solve the PDE in up to $n=10$ experts, and validate the approach with full-grid results for $n\le4$ and sparse-sector results up to $n=10$. Their computational findings challenge the global optimality of the COMB strategy for $n\ge5$, showing a unique non-COMB optimum at $n=5$ and no globally optimal strategy for $n\ge6$, while proposing concrete conjectures and highlighting open analytic questions. The work advances the ability to study high-dimensional adversarial prediction problems via PDE methods and sparse-grid techniques, with potential impact on strategy design in high-dimensional expert- Advice systems and related online-learning applications.

Abstract

This work investigates the online machine learning problem of prediction with expert advice in an adversarial setting through numerical analysis of, and experiments with, a related partial differential equation. The problem is a repeated two-person game involving decision-making at each step informed by $n$ experts in an adversarial environment. The continuum limit of this game over a large number of steps is a degenerate elliptic equation whose solution encodes the optimal strategies for both players. We develop numerical methods for approximating the solution of this equation in relatively high dimensions ($n\leq 10$) by exploiting symmetries in the equation and the solution to drastically reduce the size of the computational domain. Based on our numerical results we make a number of conjectures about the optimality of various adversarial strategies, in particular about the non-optimality of the COMB strategy.

Figures (5)

• Figure 1: Plots of the numerical solution $w$ versus the true solutions for $n=2$ and $n=3$ experts.
• Figure 2: Convergence rates and optimal strategies for $n=2,3,4$ experts, computed from the numerical solutions. The dashed red line indicates the COMB strategy, which is numerically observed to be optimal for $n=2,3,4$ experts, as the theory predicts.
• Figure 3: Numerical computation of strategy optimality for the $n=5,6$ expert problems.
• Figure 4: Numerical computation of strategy optimality for the $n=7,8$ expert problems.
• Figure 5: Numerical computation of strategy optimality for the $n=9,10$ expert problems.

Theorems & Definitions (43)

• Conjecture 1.1
• Conjecture 1.2
• Conjecture 1.3
• Remark 1.4
• Remark 1.5
• Definition 2.1
• Definition 2.2
• Theorem 2.3
• proof
• Theorem 2.4