MoE-TransMov: A Transformer-based Model for Next POI Prediction in Familiar & Unfamiliar Movements

TL;DR

The paper addresses next POI prediction by incorporating context-specific mobility through a mixture-of-experts Transformer-based model, distinguishing familiar versus unfamiliar movements without separate training. MoE-TransMov combines a shared encoder with two experts (Transformer and LSTM) and a gating network to route inputs according to history-derived cues, enabling robust performance across diverse urban contexts. Empirical results on Foursquare NYC and LY Kyoto datasets show superior ranking (Top-1/5/10) and MRR, with notable gains in unfamiliar mobility and large-scale settings, while ablation confirms the value of expert specialization and self-attention. The work advances personalized location-based services by delivering more accurate, context-aware next-POI predictions and highlights potential for cross-city transfer and richer contextual modeling.

Abstract

Accurate prediction of the next point of interest (POI) within human mobility trajectories is essential for location-based services, as it enables more timely and personalized recommendations. In particular, with the rise of these approaches, studies have shown that users exhibit different POI choices in their familiar and unfamiliar areas, highlighting the importance of incorporating user familiarity into predictive models. However, existing methods often fail to distinguish between the movements of users in familiar and unfamiliar regions. To address this, we propose MoE-TransMov, a Transformer-based model with a Transformer model with a Mixture-of-Experts (MoE) architecture designed to use one framework to capture distinct mobility patterns across different moving contexts without requiring separate training for certain data. Using user-check-in data, we classify movements into familiar and unfamiliar categories and develop a specialized expert network to improve prediction accuracy. Our approach integrates self-attention mechanisms and adaptive gating networks to dynamically select the most relevant expert models for different mobility contexts. Experiments on two real-world datasets, including the widely used but small open-source Foursquare NYC dataset and the large-scale Kyoto dataset collected with LY Corporation (Yahoo Japan Corporation), show that MoE-TransMov outperforms state-of-the-art baselines with notable improvements in Top-1, Top-5, Top-10 accuracy, and mean reciprocal rank (MRR). Given the results, we find that by using this approach, we can efficiently improve mobility predictions under different moving contexts, thereby enhancing the personalization of recommendation systems and advancing various urban applications.

Figures (4)

• Figure 1: Illustration of the definitions of familiar and unfamiliar movements, along with real-world examples and statistical results from the LY Corporation dataset. (a) Illustration of familiar movement and unfamiliar movement; (b) The footprint frequency demonstration of one real user in the LY Corporation dataset; (c) The visited POI types percentage difference in users' familiar and unfamiliar regions.
• Figure 2: Framework Overview of Our Proposed MoE-TransMov Model: Embedding, Fusion, and MoE Layers
• Figure 3: Spatial distribution of the Top-1 next POI prediction performance of the MoE-TransMov model across different regions under three evaluation settings (Unfamiliar, Familiar, and All) on the Kyoto Mobility Dataset. Darker shades indicate higher Top-1 prediction accuracy, while lighter shades represent lower accuracy.
• Figure 4: Training and validation loss convergence of different Next POI prediction models on the Kyoto Mobility Dataset.