iTIMO: An LLM-empowered Synthesis Dataset for Travel Itinerary Modification

TL;DR

The paper addresses the gap in travel recommender systems for itinerary modification by formalizing the Itinerary Perturbation Task and introducing iTIMO, a dataset generated through intent-driven perturbations using three atomic edits ($\{ADD,\; REPLACE,\; DELETE\}$) under three intents ($\{z_{pop},\; z_{dis},\; z_{div}\}$). It presents a general LLM-based data-construction pipeline that incorporates function calling and memory modules to produce high-quality, diverse need-to-modify itineraries grounded in real-world public datasets, and it introduces a hybrid evaluation metric combining macro (Hellinger distance on popularity and distance distributions) and micro (Kendall's $\tau_b$) signals with a threshold $\theta=0.1$. The authors benchmark a wide range of baselines, including eight base LLMs and two LRMs, under various training and retrieval-augmented settings, revealing surprising results such as small models with SFT sometimes outperforming larger reasoning models and that RAG+SFT does not always provide gains. They also discuss limitations, such as the dataset being user-agnostic and lacking temporal features, and they outline directions for future work, including temporal constraints and advanced RL-based fine-tuning. Overall, iTIMO provides a scalable, interpretable benchmark to study dynamic itinerary modification and to drive future TRS research toward real-time, intent-driven adaptability.

Abstract

Addressing itinerary modification is crucial for enhancing the travel experience as it is a frequent requirement during traveling. However, existing research mainly focuses on fixed itinerary planning, leaving modification underexplored. To bridge this gap, we formally define the itinerary modification task and introduce iTIMO, a dataset specifically tailored for this purpose. We identify the lack of {\itshape need-to-modify} itinerary data as the critical bottleneck hindering research on this task and propose a general pipeline to overcome it. This pipeline frames the generation of such data as an intent-driven perturbation task. It instructs large language models to perturb real world itineraries using three atomic editing operations: REPLACE, ADD, and DELETE. Each perturbation is grounded in three intents, including disruptions of popularity, spatial distance, and category diversity. Furthermore, a hybrid evaluation metric is designed to ensure perturbation effectiveness. We conduct comprehensive experiments on iTIMO, revealing the limitations of current LLMs and lead to several valuable directions for future research. Dataset and corresponding code are available at https://github.com/zelo2/iTIMO.

Figures (2)

• Figure 1: Illustrations of itinerary modification scenarios across two phases: before and during traveling. Users may proactively request to ADD POI based on subjective preferences (Phase 1), or REPLACE POI due to objective conditions such as overcrowding (Phase 2).
• Figure 2: Overall pipeline of iTIMO dataset construction. The top part introduces the process of prompt construction ( cf. Section 5.1). In the middle part, we propose V3.2 FM for high-quality itinerary perturbation ( cf. Section 5.2). In the bottom part, we manually filter noise to construct iTIMO dataset ( cf. Section 5.3).