Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Enhanced Facet Generation with LLM Editing

Joosung Lee, Jinhong Kim

TL;DR

This paper tackles query facet generation without relying on live search engine results at test time. It introduces two SERP-free strategies: multi-task learning that uses SERP as a training target to deepen query understanding, and LLM editing that refines small-model facet predictions with guidance from a large language model. By combining a fine-tuned small model (e.g., BART-base) with LLM editing, the approach achieves competitive performance relative to SERP-based methods, validated on the MIMICS dataset using both automatic and LLM-based evaluation. The work demonstrates that editing small-model outputs with LLMs can robustly align facet predictions with target distributions, offering a practical path for on-premise search clarification and wider NLP applicability.

Abstract

In information retrieval, facet identification of a user query is an important task. If a search service can recognize the facets of a user's query, it has the potential to offer users a much broader range of search results. Previous studies can enhance facet prediction by leveraging retrieved documents and related queries obtained through a search engine. However, there are challenges in extending it to other applications when a search engine operates as part of the model. First, search engines are constantly updated. Therefore, additional information may change during training and test, which may reduce performance. The second challenge is that public search engines cannot search for internal documents. Therefore, a separate search system needs to be built to incorporate documents from private domains within the company. We propose two strategies that focus on a framework that can predict facets by taking only queries as input without a search engine. The first strategy is multi-task learning to predict SERP. By leveraging SERP as a target instead of a source, the proposed model deeply understands queries without relying on external modules. The second strategy is to enhance the facets by combining Large Language Model (LLM) and the small model. Overall performance improves when small model and LLM are combined rather than facet generation individually.

Enhanced Facet Generation with LLM Editing

TL;DR

This paper tackles query facet generation without relying on live search engine results at test time. It introduces two SERP-free strategies: multi-task learning that uses SERP as a training target to deepen query understanding, and LLM editing that refines small-model facet predictions with guidance from a large language model. By combining a fine-tuned small model (e.g., BART-base) with LLM editing, the approach achieves competitive performance relative to SERP-based methods, validated on the MIMICS dataset using both automatic and LLM-based evaluation. The work demonstrates that editing small-model outputs with LLMs can robustly align facet predictions with target distributions, offering a practical path for on-premise search clarification and wider NLP applicability.

Abstract

In information retrieval, facet identification of a user query is an important task. If a search service can recognize the facets of a user's query, it has the potential to offer users a much broader range of search results. Previous studies can enhance facet prediction by leveraging retrieved documents and related queries obtained through a search engine. However, there are challenges in extending it to other applications when a search engine operates as part of the model. First, search engines are constantly updated. Therefore, additional information may change during training and test, which may reduce performance. The second challenge is that public search engines cannot search for internal documents. Therefore, a separate search system needs to be built to incorporate documents from private domains within the company. We propose two strategies that focus on a framework that can predict facets by taking only queries as input without a search engine. The first strategy is multi-task learning to predict SERP. By leveraging SERP as a target instead of a source, the proposed model deeply understands queries without relying on external modules. The second strategy is to enhance the facets by combining Large Language Model (LLM) and the small model. Overall performance improves when small model and LLM are combined rather than facet generation individually.
Paper Structure (22 sections, 3 equations, 1 figure, 12 tables)

This paper contains 22 sections, 3 equations, 1 figure, 12 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Task Definition
  4. Method
  5. Multi-task Learning
  6. LLM Editing
  7. Editing Prompt.
  8. Experiments
  9. Dataset
  10. Evaluation Metric
  11. Automatic Evaluation
  12. LLM-based Evaluation
  13. Result and Discussion
  14. Comparison with Previous Methods
  15. Result of LLM-based Evaluation
  16. ...and 7 more sections

Figures (1)

  • Figure 1: Overview of previous methods and our method.