Testing the Feasibility of Linear Programs with Bandit Feedback

TL;DR

This work addresses the problem in the linear bandit setting, thus characterising the costs of feasibility testing for an unknown linear program using bandit feedback, and constructs a novel test based on low-regret algorithms and a nonasymptotic law of iterated logarithms that adapts to the `signal level,' $\Gamma,$ of any instance.

Abstract

While the recent literature has seen a surge in the study of constrained bandit problems, all existing methods for these begin by assuming the feasibility of the underlying problem. We initiate the study of testing such feasibility assumptions, and in particular address the problem in the linear bandit setting, thus characterising the costs of feasibility testing for an unknown linear program using bandit feedback. Concretely, we test if $\exists x: Ax \ge 0$ for an unknown $A \in \mathbb{R}^{m \times d}$, by playing a sequence of actions $x_t\in \mathbb{R}^d$, and observing $Ax_t + \mathrm{noise}$ in response. By identifying the hypothesis as determining the sign of the value of a minimax game, we construct a novel test based on low-regret algorithms and a nonasymptotic law of iterated logarithms. We prove that this test is reliable, and adapts to the `signal level,' $Γ,$ of any instance, with mean sample costs scaling as $\widetilde{O}(d^2/Γ^2)$. We complement this by a minimax lower bound of $Ω(d/Γ^2)$ for sample costs of reliable tests, dominating prior asymptotic lower bounds by capturing the dependence on $d$, and thus elucidating a basic insight missing in the extant literature on such problems.

Figures (2)

• Figure 1: Illustration of the Signal Level. The ball is $\mathcal{X},$ and lines with arrows indicate the feasible half spaces for each constraint, and assuming that $\|A^i\|=1$ for all $i$. Left. A feasible case; $\Gamma > 0$ is the distance of the marked point from the constraints, i.e., the length of the red dash-dotted line. Right. An infeasible case with $-\Gamma > 0$ shown similarly.
• Figure 2: Behaviour of the stopping time as $d$ is varied for fixed $\Gamma = 1/\sqrt{2}$ (left) and $\Gamma$ is varied for fixed $d = 4$ (right) over the unit ball with $m = 2$. Averages and one-sigma error bars over 50 runs are reported. The test never returned an incorrect hypothesis. Notice the sharp advantage of $\tau_{\mathrm{early}}$ in feasible cases, in that it is about a factor of $10$ smaller than $\tau$. (best viewed zoomed-in)

Theorems & Definitions (45)

• Definition 1
• Definition 2
• Definition 4
• Lemma 5
• Lemma 6
• Lemma 7
• Theorem 8
• Lemma 9
• Theorem 10
• Lemma 11