BanditLP: Large-Scale Stochastic Optimization for Personalized Recommendations

TL;DR

BanditLP addresses multi-stakeholder contextual bandits by coupling neural Thompson Sampling with a large-scale linear program to maximize a primary reward while hard-constraining platform-, provider-, and user-level objectives. The framework decouples learning from optimization and leverages a scalable LP solver (DuaLip) to operate at web scale, with neural TS providing uncertainty estimates via a Laplace-approximation and temperature-controlled exploration. It is evaluated on synthetic and public benchmarks and deployed in LinkedIn's email marketing system, where online A/B testing shows a meaningful revenue uplift and reduction in unsubscribe rates, validating the practical impact of integrated exploration and constrained optimization. The work also provides deployment insights, including probability calibration, exploration tuning, LP convergence, and data-diverted experiment designs to avoid leakage, while noting limitations such as interaction effects across units and the need for real-time online learning in future work.

Abstract

We present BanditLP, a scalable multi-stakeholder contextual bandit framework that unifies neural Thompson Sampling for learning objective-specific outcomes with a large-scale linear program for constrained action selection at serving time. The methodology is application-agnostic, compatible with arbitrary neural architectures, and deployable at web scale, with an LP solver capable of handling billions of variables. Experiments on public benchmarks and synthetic data show consistent gains over strong baselines. We apply this approach in LinkedIn's email marketing system and demonstrate business win, illustrating the value of integrated exploration and constrained optimization in production.

Figures (7)

• Figure 1: Overview of email marketing recommender system based on BanditLP at LinkedIn.
• Figure 2: Calibration heuristic preserves exploration.
• Figure 3: Synthetic experiments. Top: cumulative reward over rounds. Middle: global constraint violation (no violation if $\le 0$). Bottom: provider-level constraint violations across item sets (no violation if $\le 0$). Shaded regions show 95% CIs over 50 runs.
• Figure 4: Open Bandit Dataset (imputed matrix) experiments. Top: cumulative reward over rounds. Middle: global constraint violation (no violation if $\le 0$). Bottom: provider-level constraint violations across item sets (no violation if $\le 0$). Shaded regions show 95% CIs over 20 runs.
• Figure 5: Data-diverted experiment setup to measure benefits of exploration.