Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Large Language Models for Next Point-of-Interest Recommendation

Peibo Li, Maarten de Rijke, Hao Xue, Shuang Ao, Yang Song, Flora D. Salim

TL;DR

This work tackles next POI recommendation by leveraging pretrained LLMs to preserve and exploit rich contextual information in LBSN data. The authors introduce LLM4POI, a framework built on trajectory prompting, a novel key-query similarity mechanism, and supervised fine-tuning with parameter-efficient methods (LoRA), quantization (NF4), and long-context support (FlashAttention-2). Empirical results across three real-world datasets show substantial improvements over strong baselines, with notable gains for cold-start and short-trajectory scenarios and evidence of cross-dataset generalization. Limitations include computational efficiency and context-length constraints, which the authors address as directions for future work such as chain-of-thought reasoning and extending beyond single-item recommendations.

Abstract

The next Point of Interest (POI) recommendation task is to predict users' immediate next POI visit given their historical data. Location-Based Social Network (LBSN) data, which is often used for the next POI recommendation task, comes with challenges. One frequently disregarded challenge is how to effectively use the abundant contextual information present in LBSN data. Previous methods are limited by their numerical nature and fail to address this challenge. In this paper, we propose a framework that uses pretrained Large Language Models (LLMs) to tackle this challenge. Our framework allows us to preserve heterogeneous LBSN data in its original format, hence avoiding the loss of contextual information. Furthermore, our framework is capable of comprehending the inherent meaning of contextual information due to the inclusion of commonsense knowledge. In experiments, we test our framework on three real-world LBSN datasets. Our results show that the proposed framework outperforms the state-of-the-art models in all three datasets. Our analysis demonstrates the effectiveness of the proposed framework in using contextual information as well as alleviating the commonly encountered cold-start and short trajectory problems.

Large Language Models for Next Point-of-Interest Recommendation

TL;DR

This work tackles next POI recommendation by leveraging pretrained LLMs to preserve and exploit rich contextual information in LBSN data. The authors introduce LLM4POI, a framework built on trajectory prompting, a novel key-query similarity mechanism, and supervised fine-tuning with parameter-efficient methods (LoRA), quantization (NF4), and long-context support (FlashAttention-2). Empirical results across three real-world datasets show substantial improvements over strong baselines, with notable gains for cold-start and short-trajectory scenarios and evidence of cross-dataset generalization. Limitations include computational efficiency and context-length constraints, which the authors address as directions for future work such as chain-of-thought reasoning and extending beyond single-item recommendations.

Abstract

The next Point of Interest (POI) recommendation task is to predict users' immediate next POI visit given their historical data. Location-Based Social Network (LBSN) data, which is often used for the next POI recommendation task, comes with challenges. One frequently disregarded challenge is how to effectively use the abundant contextual information present in LBSN data. Previous methods are limited by their numerical nature and fail to address this challenge. In this paper, we propose a framework that uses pretrained Large Language Models (LLMs) to tackle this challenge. Our framework allows us to preserve heterogeneous LBSN data in its original format, hence avoiding the loss of contextual information. Furthermore, our framework is capable of comprehending the inherent meaning of contextual information due to the inclusion of commonsense knowledge. In experiments, we test our framework on three real-world LBSN datasets. Our results show that the proposed framework outperforms the state-of-the-art models in all three datasets. Our analysis demonstrates the effectiveness of the proposed framework in using contextual information as well as alleviating the commonly encountered cold-start and short trajectory problems.
Paper Structure (26 sections, 6 equations, 8 figures, 10 tables)

This paper contains 26 sections, 6 equations, 8 figures, 10 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Next POI Recommendation
  4. LLM for Time-series Data
  5. LLM for Recommender Systems
  6. Problem Definition
  7. Methodology
  8. Trajectory Prompting
  9. Key-Query Pair Similarity
  10. Supervised Fine-tuning
  11. Experiment
  12. Experimental Setup
  13. Datasets
  14. Baselines.
  15. Our Models
  16. ...and 11 more sections

Figures (8)

  • Figure 1: Comparison of two paradigms for the next POI task: (a) a typical feature transformation paradigm and (b) the proposed language model-based paradigm.
  • Figure 2: Our overall LLM-based framework for next POI recommendation.
  • Figure 3: The process of forming and pairing key and query prompts. Each trajectory is made into a key prompt and a query prompt. The key prompt contains the check-in records excluding the last entry of the trajectory, while the query prompt contains the entire trajectory. A key prompt is paired with every query prompt representing the trajectories before the current trajectory.
  • Figure 4: Similarity computation for each pair of key and query. Each pair of key and query prompts is fed into a pre-trained LLAMA2 separately. We use the last hidden layer embeddings to compute their cosine similarity.
  • Figure 5: For data in the training set, the <question> part in the prompt is fed into the pre-trained LLM, and the <answer> part is used to guide the fine-tuning.
  • ...and 3 more figures