Truthful mechanisms for linear bandit games with private contexts

TL;DR

This work addresses private context misreporting in Bayesian contextual linear bandits, identifying that both deterministic and stochastic standard algorithms fail to ensure truthful reporting under misreporting. It introduces a truthful Thompson sampling mechanism that solves a per-step linear program to enforce incentive compatibility while staying close to standard Thompson sampling, preserving favorable regret behavior. Theoretical results establish an $O(\ln T)$ frequentist regret bound in the two-context Gaussian setting (same or different optimal arms analyzed separately), and simulations demonstrate sublinear regret across multiple contexts and sub-Gaussian noise distributions. The approach provides a principled mechanism design for incentive-compatible personalized decision-making in sequential allocation problems, with practical implications for adaptive clinical trials and personalized recommendations.

Abstract

The contextual bandit problem, where agents arrive sequentially with personal contexts and the system adapts its arm allocation decisions accordingly, has recently garnered increasing attention for enabling more personalized outcomes. However, in many healthcare and recommendation applications, agents have private profiles and may misreport their contexts to gain from the system. For example, in adaptive clinical trials, where hospitals sequentially recruit volunteers to test multiple new treatments and adjust plans based on volunteers' reported profiles such as symptoms and interim data, participants may misreport severe side effects like allergy and nausea to avoid perceived suboptimal treatments. We are the first to study this issue of private context misreporting in a stochastic contextual bandit game between the system and non-repeated agents. We show that traditional low-regret algorithms, such as UCB family algorithms and Thompson sampling, fail to ensure truthful reporting and can result in linear regret in the worst case, while traditional truthful algorithms like explore-then-commit (ETC) and $ε$-greedy algorithm incur sublinear but high regret. We propose a mechanism that uses a linear program to ensure truthfulness while minimizing deviation from Thompson sampling, yielding an $O(\ln T)$ frequentist regret. Our numerical experiments further demonstrate strong performance in multiple contexts and across other distribution families.

Figures (5)

• Figure 1: Geometric illustration of context $\chi_2$'s incentive to misreport as $\chi_1$.
• Figure 2: Initial stage for contexts with different optimal arms
• Figure 3: Converging stage for contexts with different optimal arms
• Figure 4: Cumulative regret at each time step under our truthful-TS mechanism in Mechanism \ref{['A1']} and Thompson sampling algorithm, and Gaussian and Laplace noises with 2 contexts, 2 arms, and 2 dimensions.
• Figure 5: Cumulative regret at each time step under our Mechanism \ref{['A1']} (T-TS) and Thompson sampling algorithm (TS), where the number of contexts varies between $N\in\{3,5,9\}$ and $d\in\{5,9\}$.

Theorems & Definitions (15)

• definition 1
• lemma 1
• lemma 2
• theorem 1
• theorem 2
• lemma 8
• lemma 9
• lemma 10
• lemma 11
• lemma 12