Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Conversational Dueling Bandits in Generalized Linear Models

Shuhua Yang, Hui Yuan, Xiaoying Zhang, Mengdi Wang, Hong Zhang, Huazheng Wang

TL;DR

This paper introduces relative feedback-based conversations into conversational recommendation systems through the integration of dueling bandits in generalized linear models (GLM) and proposes a novel conversational dueling bandit algorithm called ConDuel, which demonstrates the efficacy of the proposed framework and has the potential to extend to multinomial logit bandits with theoretical and experimental guarantees.

Abstract

Conversational recommendation systems elicit user preferences by interacting with users to obtain their feedback on recommended commodities. Such systems utilize a multi-armed bandit framework to learn user preferences in an online manner and have received great success in recent years. However, existing conversational bandit methods have several limitations. First, they only enable users to provide explicit binary feedback on the recommended items or categories, leading to ambiguity in interpretation. In practice, users are usually faced with more than one choice. Relative feedback, known for its informativeness, has gained increasing popularity in recommendation system design. Moreover, current contextual bandit methods mainly work under linear reward assumptions, ignoring practical non-linear reward structures in generalized linear models. Therefore, in this paper, we introduce relative feedback-based conversations into conversational recommendation systems through the integration of dueling bandits in generalized linear models (GLM) and propose a novel conversational dueling bandit algorithm called ConDuel. Theoretical analyses of regret upper bounds and empirical validations on synthetic and real-world data underscore ConDuel's efficacy. We also demonstrate the potential to extend our algorithm to multinomial logit bandits with theoretical and experimental guarantees, which further proves the applicability of the proposed framework.

Conversational Dueling Bandits in Generalized Linear Models

TL;DR

This paper introduces relative feedback-based conversations into conversational recommendation systems through the integration of dueling bandits in generalized linear models (GLM) and proposes a novel conversational dueling bandit algorithm called ConDuel, which demonstrates the efficacy of the proposed framework and has the potential to extend to multinomial logit bandits with theoretical and experimental guarantees.

Abstract

Conversational recommendation systems elicit user preferences by interacting with users to obtain their feedback on recommended commodities. Such systems utilize a multi-armed bandit framework to learn user preferences in an online manner and have received great success in recent years. However, existing conversational bandit methods have several limitations. First, they only enable users to provide explicit binary feedback on the recommended items or categories, leading to ambiguity in interpretation. In practice, users are usually faced with more than one choice. Relative feedback, known for its informativeness, has gained increasing popularity in recommendation system design. Moreover, current contextual bandit methods mainly work under linear reward assumptions, ignoring practical non-linear reward structures in generalized linear models. Therefore, in this paper, we introduce relative feedback-based conversations into conversational recommendation systems through the integration of dueling bandits in generalized linear models (GLM) and propose a novel conversational dueling bandit algorithm called ConDuel. Theoretical analyses of regret upper bounds and empirical validations on synthetic and real-world data underscore ConDuel's efficacy. We also demonstrate the potential to extend our algorithm to multinomial logit bandits with theoretical and experimental guarantees, which further proves the applicability of the proposed framework.
Paper Structure (24 sections, 10 theorems, 34 equations, 4 figures, 2 algorithms)

This paper contains 24 sections, 10 theorems, 34 equations, 4 figures, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related work
  3. Conversational Bandits.
  4. Utility-based Dueling Bandits.
  5. Problem Formulation
  6. Generalized Linear Dueling Feedback.
  7. Conversation on Key-Terms and Frequency
  8. Cumulative Regret.
  9. Conversational Dueling Bandits with GLM and Regret Analysis
  10. ConDuel Algorithm
  11. Parameter Estimation
  12. Arm Selection Module
  13. Key-term Selection Module
  14. Regret Upper Bound
  15. Experiments
  16. ...and 9 more sections

Key Result

Lemma 1

Assume $\epsilon_t$ and $\Tilde{\epsilon}_t$ defined in Eq. eq:3 and eq:4 are conditional $R$-sub-Gaussian, $d_t$ is denoted as the difference contextual vectors for the selected arm pair $(x_t, x_t^{\prime})$. Then for any $d_{t}\in\mathcal{D}_t$, with probability at least $(1-\delta)$, we have the where $\alpha_t = \frac{2}{\kappa_1}(R\sqrt{d\log((1+\frac{4\kappa_1(t+b(t))}{d\lambda})/\sigma)}+\

Figures (4)

  • Figure 1: An illustrative example of a conversational system with pairwise feedback compared with a system that only allows explicit feedback.
  • Figure 2: Cumulative regret on synthetic and real-world datasets
  • Figure 3: Ablation study on synthetic data
  • Figure 4: Cumulative regret on synthetic and real-world datasets

Theorems & Definitions (10)

  • Lemma 1
  • Lemma 2
  • Lemma 3
  • Theorem 1
  • Theorem 2
  • Lemma 4
  • Lemma 5
  • Lemma 6
  • Lemma 7
  • Lemma 8