Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Data Synthesis Method Driven by Large Language Models for Proactive Mining of Implicit User Intentions in Tourism

Jinqiang Wang, Huansheng Ning, Tao Zhu, Jianguo Ding

TL;DR

This work tackles the challenge of proactively mining implicit user intentions in tourism, where LLMs often fail to request clarifications in ambiguous scenarios. It introduces SynPT, an LLM-driven data synthesis framework that builds a tourism-centric seed data pool from Chinese travel sites and uses a dual-agent dialogue system with memory and emotion-aware reasoning to generate SynPT-Dialog data. The synthesized data trains a smaller model, Qwen-PT, to proactively infer and summarize user intentions, including emotion and intention-value reasoning, achieving strong improvements over baselines on multiple metrics and demonstrating transfer to English-language settings. The approach yields practical benefits for proactive support in travel assistants and provides open-source tools and datasets to facilitate further research and application in the domain.

Abstract

In the tourism domain, Large Language Models (LLMs) often struggle to mine implicit user intentions from tourists' ambiguous inquiries and lack the capacity to proactively guide users toward clarifying their needs. A critical bottleneck is the scarcity of high-quality training datasets that facilitate proactive questioning and implicit intention mining. While recent advances leverage LLM-driven data synthesis to generate such datasets and transfer specialized knowledge to downstream models, existing approaches suffer from several shortcomings: (1) lack of adaptation to the tourism domain, (2) skewed distributions of detail levels in initial inquiries, (3) contextual redundancy in the implicit intention mining module, and (4) lack of explicit thinking about tourists' emotions and intention values. Therefore, we propose SynPT (A Data Synthesis Method Driven by LLMs for Proactive Mining of Implicit User Intentions in the Tourism), which constructs an LLM-driven user agent and assistant agent to simulate dialogues based on seed data collected from Chinese tourism websites. This approach addresses the aforementioned limitations and generates SynPT-Dialog, a training dataset containing explicit reasoning. The dataset is utilized to fine-tune a general LLM, enabling it to proactively mine implicit user intentions. Experimental evaluations, conducted from both human and LLM perspectives, demonstrate the superiority of SynPT compared to existing methods. Furthermore, we analyze key hyperparameters and present case studies to illustrate the practical applicability of our method, including discussions on its adaptability to English-language scenarios. All code and data are publicly available.

A Data Synthesis Method Driven by Large Language Models for Proactive Mining of Implicit User Intentions in Tourism

TL;DR

This work tackles the challenge of proactively mining implicit user intentions in tourism, where LLMs often fail to request clarifications in ambiguous scenarios. It introduces SynPT, an LLM-driven data synthesis framework that builds a tourism-centric seed data pool from Chinese travel sites and uses a dual-agent dialogue system with memory and emotion-aware reasoning to generate SynPT-Dialog data. The synthesized data trains a smaller model, Qwen-PT, to proactively infer and summarize user intentions, including emotion and intention-value reasoning, achieving strong improvements over baselines on multiple metrics and demonstrating transfer to English-language settings. The approach yields practical benefits for proactive support in travel assistants and provides open-source tools and datasets to facilitate further research and application in the domain.

Abstract

In the tourism domain, Large Language Models (LLMs) often struggle to mine implicit user intentions from tourists' ambiguous inquiries and lack the capacity to proactively guide users toward clarifying their needs. A critical bottleneck is the scarcity of high-quality training datasets that facilitate proactive questioning and implicit intention mining. While recent advances leverage LLM-driven data synthesis to generate such datasets and transfer specialized knowledge to downstream models, existing approaches suffer from several shortcomings: (1) lack of adaptation to the tourism domain, (2) skewed distributions of detail levels in initial inquiries, (3) contextual redundancy in the implicit intention mining module, and (4) lack of explicit thinking about tourists' emotions and intention values. Therefore, we propose SynPT (A Data Synthesis Method Driven by LLMs for Proactive Mining of Implicit User Intentions in the Tourism), which constructs an LLM-driven user agent and assistant agent to simulate dialogues based on seed data collected from Chinese tourism websites. This approach addresses the aforementioned limitations and generates SynPT-Dialog, a training dataset containing explicit reasoning. The dataset is utilized to fine-tune a general LLM, enabling it to proactively mine implicit user intentions. Experimental evaluations, conducted from both human and LLM perspectives, demonstrate the superiority of SynPT compared to existing methods. Furthermore, we analyze key hyperparameters and present case studies to illustrate the practical applicability of our method, including discussions on its adaptability to English-language scenarios. All code and data are publicly available.
Paper Structure (33 sections, 20 equations, 4 figures, 10 tables)

This paper contains 33 sections, 20 equations, 4 figures, 10 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Data synthesis methods driven by LLMs
  4. Proactive user intention mining
  5. Methods
  6. Overview
  7. Seed data pool
  8. Initial inquiry
  9. User intention thinking
  10. User emotion thinking
  11. Reference options thinking
  12. Experiment setup
  13. Compared methods
  14. Training and test sets
  15. Metrics
  16. ...and 18 more sections

Figures (4)

  • Figure 1: Various Methods of Data Synthesis. (a) This method utilizes a single agent to synthesize data. a representative work is xu2023baize. (b) This method employs two interacting agents for data synthesis. a representative work is qian2024tell. (c) In comparison to (b), this approach introduces emotion thinking and option thinking modules, and further optimizes the mechanisms of the initial inquiry and intention thinking modules. In (a) and (b), each agent is played by a single LLM. In (c), each module is played by a single LLM, and each agent consists of multiple such modules.
  • Figure 2: Overview of SynPT Methodology and Downstream Processing. It comprises four components: the seed data pool, the user agent, the assistant agent, and the recording component. The recording component collects dialogue data generated by interactions between the user agent and the assistant agent, forming a multi-turn dialogue dataset capable of proactively mine implicit user intention. The Qwen model is subsequently fine-tuned on this dataset, resulting in Qwen-PT.
  • Figure 3: Impact of Using Different LLMs for the User Agent and the Assistant Agent
  • Figure 4: Violin plot of word length distributions in initial inquiries generated by SynPT with and without the probability control (PC) strategy.