Tractable Multinomial Logit Contextual Bandits with Non-Linear Utilities

TL;DR

This work tackles contextual MNL bandits with non-linear utilities by proposing ONL-MNL, a UCB-based algorithm that combines a pilot estimator with an optimistic exploration phase. Under a realizability assumption and a generalized geometric condition, the method achieves a near-optimal regret of \(\widetilde{\mathcal{O}}(\sqrt{T})\) that is independent of the number of items \(N\), while remaining computationally tractable and NTK-free. The approach avoids NTK over-parameterization, instead leveraging a Taylor-linearization of the non-linear utility and a novel concentration inequality for the parameter estimator. Empirical results demonstrate robustness to misspecification and favorable scaling with the item pool, validating its practical applicability in complex recommendation and assortment settings.

Abstract

We study the multinomial logit (MNL) contextual bandit problem for sequential assortment selection. Although most existing research assumes utility functions to be linear in item features, this linearity assumption restricts the modeling of intricate interactions between items and user preferences. A recent work (Zhang & Luo, 2024) has investigated general utility function classes, yet its method faces fundamental trade-offs between computational tractability and statistical efficiency. To address this limitation, we propose a computationally efficient algorithm for MNL contextual bandits leveraging the upper confidence bound principle, specifically designed for non-linear parametric utility functions, including those modeled by neural networks. Under a realizability assumption and a mild geometric condition on the utility function class, our algorithm achieves a regret bound of $\tilde{O}(\sqrt{T})$, where $T$ denotes the total number of rounds. Our result establishes that sharp $\tilde{O}(\sqrt{T})$-regret is attainable even with neural network-based utilities, without relying on strong assumptions such as neural tangent kernel approximations. To the best of our knowledge, our proposed method is the first computationally tractable algorithm for MNL contextual bandits with non-linear utilities that provably attains $\tilde{O}(\sqrt{T})$ regret. Comprehensive numerical experiments validate the effectiveness of our approach, showing robust performance not only in realizable settings but also in scenarios with model misspecification.

Figures (7)

• Figure 1: Example of a highly non-convex loss $\ell_{\mathrm{sq}}(\mathbf{w})$ that satisfies Assumption \ref{['assm:generalized geometric condition']}. The solid line shows the lower bound given by the pointwise minimum of the strong convexity and growth conditions over the equivalence set $\mathcal{W}^*$. For neural networks, the squared loss may have multiple global minima due to parameter symmetries or over-parameterization. This highlights that Assumption \ref{['assm:generalized geometric condition']} is strictly weaker than prior geometric conditions, enabling it to cover a wider range of utility functions.
• Figure 2: Cumulative regret comparison between $\texttt{ONL-MNL}$ (ours) and baselines under Gaussian contexts. The results for the uniform context distribution are provided in Figure \ref{['fig:main exp_uniform']}.
• Figure 3: Cumulative regret comparison between $\texttt{ONL-MNL}$ (ours) and $\varepsilon \texttt{-greedy-MNL}$zhang2024contextual under varying number of items $N$.
• Figure 4: Cumulative regret comparison between $\texttt{ONL-MNL}$ (ours) and baselines under realizable and misspecified settings with uniform context distributions.
• Figure 5: Cumulative regret comparison between $\texttt{ONL-MNL}$ (ours) and baseline methods in the semi-synthetic experimental setting based on the IMDB Large Movie Review dataset maas2011learning.

Theorems & Definitions (18)

• Theorem 1: Regret Bound of $\texttt{ONL-MNL}$
• Remark 1
• Lemma 1: Convergence rate of $\hat{\mathbf{w}}_0$
• Lemma 2: Confidence set
• Lemma 3: Optimistic utility
• Lemma 4: Reverse Lipschitzness of the MNL model
• Lemma 5
• Lemma 6
• Lemma 7
• Lemma 8