Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Improving Sequential Query Recommendation with Immediate User Feedback

Shameem A Puthiya Parambath, Christos Anagnostopoulos, Roderick Murray-Smith

TL;DR

The paper tackles next-query recommendation in interactive knowledge-discovery tasks by modeling feedback as an adversarial, non-stochastic MAB with a dynamic, context-dependent action set. It introduces TEF, a non-stochastic MAB that ensembles transformer-based autoregressive experts to build a growing candidate set $\mathcal{C}_t$ and samples from a learned distribution $p_t$ using immediate user rewards. A theoretical regret bound of $\mathcal{O}\left(\sqrt{\lvert \mathcal{C}_T\rvert / T}\right)$ is proven, and empirical evaluation on a large-scale query log demonstrates substantial improvements over single-expert transformer baselines, with fine-tuned CLMs outperforming MLMs. The work highlights the practical impact of integrating real-time user feedback with transformer ensembles for sequential query prediction and provides open-source data and code for reproducibility.

Abstract

We propose an algorithm for next query recommendation in interactive data exploration settings, like knowledge discovery for information gathering. The state-of-the-art query recommendation algorithms are based on sequence-to-sequence learning approaches that exploit historical interaction data. Due to the supervision involved in the learning process, such approaches fail to adapt to immediate user feedback. We propose to augment the transformer-based causal language models for query recommendations to adapt to the immediate user feedback using multi-armed bandit (MAB) framework. We conduct a large-scale experimental study using log files from a popular online literature discovery service and demonstrate that our algorithm improves the per-round regret substantially, with respect to the state-of-the-art transformer-based query recommendation models, which do not make use of immediate user feedback. Our data model and source code are available at https://github.com/shampp/exp3_ss

Improving Sequential Query Recommendation with Immediate User Feedback

TL;DR

The paper tackles next-query recommendation in interactive knowledge-discovery tasks by modeling feedback as an adversarial, non-stochastic MAB with a dynamic, context-dependent action set. It introduces TEF, a non-stochastic MAB that ensembles transformer-based autoregressive experts to build a growing candidate set and samples from a learned distribution using immediate user rewards. A theoretical regret bound of is proven, and empirical evaluation on a large-scale query log demonstrates substantial improvements over single-expert transformer baselines, with fine-tuned CLMs outperforming MLMs. The work highlights the practical impact of integrating real-time user feedback with transformer ensembles for sequential query prediction and provides open-source data and code for reproducibility.

Abstract

We propose an algorithm for next query recommendation in interactive data exploration settings, like knowledge discovery for information gathering. The state-of-the-art query recommendation algorithms are based on sequence-to-sequence learning approaches that exploit historical interaction data. Due to the supervision involved in the learning process, such approaches fail to adapt to immediate user feedback. We propose to augment the transformer-based causal language models for query recommendations to adapt to the immediate user feedback using multi-armed bandit (MAB) framework. We conduct a large-scale experimental study using log files from a popular online literature discovery service and demonstrate that our algorithm improves the per-round regret substantially, with respect to the state-of-the-art transformer-based query recommendation models, which do not make use of immediate user feedback. Our data model and source code are available at https://github.com/shampp/exp3_ss
Paper Structure (15 sections, 1 theorem, 11 equations, 5 figures, 3 tables)

This paper contains 15 sections, 1 theorem, 11 equations, 5 figures, 3 tables.

Table of Contents

  1. Introduction
  2. The Query Prediction Problem in Multi-arm Bandits Context
  3. Contribution
  4. Related Work
  5. Autoregressive & Causal Language Models for Query Prediction
  6. Multi-armed Bandit Systems for Query Prediction
  7. Ensemble-based Sequential Query Prediction with Immediate Feedback
  8. Non-stochastic MAB with Transformer-based Experts and Feedback
  9. TEF Regret Analysis
  10. Experimental Evaluation
  11. Datasets & Protocol
  12. Performance Metrics
  13. Baseline Approaches & Models under Comparison
  14. Experimental Results & Analysis
  15. Conclusions

Key Result

Theorem 1

For any $T>0$, $\lvert\mathcal{C}_{T}\rvert>0$, and learning rate $\eta = \min(0.5, \sqrt{\frac{T+1}{T(2\lvert \mathcal{C}_T \rvert +1)}})$, the regret of the TEF algorithm is $\mathcal{O}\left(\sqrt{\frac{\lvert \mathcal{C}_T \rvert}{T}}\right)$.

Figures (5)

  • Figure 1: Per-round regret of different comparison algorithms and baselines using pre-trained models.
  • Figure 2: Per-round regret of different comparison algorithms and baselines using models learned from scratch.
  • Figure 3: Per-round regret of TEF using both fine-tuned and trained-from-scratch experts.
  • Figure 4: Increase the rate of the candidate set cardinality vs. rounds (using $k=2$ query predictions per expert).
  • Figure 5: The per-round regret for different values of $k$ in TEF .

Theorems & Definitions (4)

  • Theorem 1
  • proof
  • Remark 1
  • Remark 2