Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Trigger$^3$: Refining Query Correction via Adaptive Model Selector

Kepu Zhang, Zhongxiang Sun, Xiao Zhang, Xiaoxue Zang, Kai Zheng, Yang Song, Jun Xu

TL;DR

This work tackles the challenge of correcting erroneous search queries by combining small, efficient correction models with powerful LLMs. It introduces Trigger$^3$, a three-trigger framework that adaptively routes queries through a Correction Trigger, an LLM Trigger, and a Fallback Trigger to balance accuracy and efficiency. Empirical results on Commercial and QQ datasets show Trigger$^3$ outperforms standalone small models and LLM-based baselines, while maintaining high efficiency due to measured LLM usage. The approach advances practical query correction by leveraging model complementarities and principled decision mechanisms, with potential impact on production search systems.

Abstract

In search scenarios, user experience can be hindered by erroneous queries due to typos, voice errors, or knowledge gaps. Therefore, query correction is crucial for search engines. Current correction models, usually small models trained on specific data, often struggle with queries beyond their training scope or those requiring contextual understanding. While the advent of Large Language Models (LLMs) offers a potential solution, they are still limited by their pre-training data and inference cost, particularly for complex queries, making them not always effective for query correction. To tackle these, we propose Trigger$^3$, a large-small model collaboration framework that integrates the traditional correction model and LLM for query correction, capable of adaptively choosing the appropriate correction method based on the query and the correction results from the traditional correction model and LLM. Trigger$^3$ first employs a correction trigger to filter out correct queries. Incorrect queries are then corrected by the traditional correction model. If this fails, an LLM trigger is activated to call the LLM for correction. Finally, for queries that no model can correct, a fallback trigger decides to return the original query. Extensive experiments demonstrate Trigger$^3$ outperforms correction baselines while maintaining efficiency.

Trigger$^3$: Refining Query Correction via Adaptive Model Selector

TL;DR

This work tackles the challenge of correcting erroneous search queries by combining small, efficient correction models with powerful LLMs. It introduces Trigger, a three-trigger framework that adaptively routes queries through a Correction Trigger, an LLM Trigger, and a Fallback Trigger to balance accuracy and efficiency. Empirical results on Commercial and QQ datasets show Trigger outperforms standalone small models and LLM-based baselines, while maintaining high efficiency due to measured LLM usage. The approach advances practical query correction by leveraging model complementarities and principled decision mechanisms, with potential impact on production search systems.

Abstract

In search scenarios, user experience can be hindered by erroneous queries due to typos, voice errors, or knowledge gaps. Therefore, query correction is crucial for search engines. Current correction models, usually small models trained on specific data, often struggle with queries beyond their training scope or those requiring contextual understanding. While the advent of Large Language Models (LLMs) offers a potential solution, they are still limited by their pre-training data and inference cost, particularly for complex queries, making them not always effective for query correction. To tackle these, we propose Trigger, a large-small model collaboration framework that integrates the traditional correction model and LLM for query correction, capable of adaptively choosing the appropriate correction method based on the query and the correction results from the traditional correction model and LLM. Trigger first employs a correction trigger to filter out correct queries. Incorrect queries are then corrected by the traditional correction model. If this fails, an LLM trigger is activated to call the LLM for correction. Finally, for queries that no model can correct, a fallback trigger decides to return the original query. Extensive experiments demonstrate Trigger outperforms correction baselines while maintaining efficiency.

Paper Structure

This paper contains 26 sections, 10 equations, 4 figures, 6 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Query Correction in Search Engines
  4. Model Selection of Language Models
  5. Trigger$^3$: The Proposed Framework
  6. Task Formalization
  7. General Framework
  8. The First Trigger: Correction Trigger
  9. The Second Trigger: LLM Trigger
  10. The Third Trigger: Fallback Trigger
  11. Model Training in $\mathrm{Trigger}^3$
  12. Experiments
  13. Experimental Settings
  14. Dataset
  15. Metrics
  16. ...and 11 more sections

Figures (4)

  • Figure 1: Examples of query correction, where the red characters are the original errors, the blue characters are the results of corrected but incorrect, and the green characters are the correct result. The small model is traditional correction model GECToR and the LLM is Qwen1.5-7B-Chat.
  • Figure 2: The architecture of the proposed framework $\mathrm{Trigger}^3$. (a) The general framework of $\mathrm{Trigger}^3$. (b) The Illustration of Correction Trigger (CT). (c) The Illustration of LLM Trigger (LT). (d) The Illustration of Fallback Trigger (FT).
  • Figure 3: Average LLM Coverage of $\mathrm{Trigger}^3$ and the three frameworks when the LLM is Qwen1.5-7B-Chat. The lower the bar, the better.
  • Figure 4: LLM Templates of Zero-shot, Few-shot and Few-shot CoT.