RHYTHM: Reasoning with Hierarchical Temporal Tokenization for Human Mobility

TL;DR

RHYTHM tackles human mobility prediction by unifying hierarchical temporal tokenization with frozen LLM reasoning to capture multi-scale daily and weekly rhythms. It tokenizes trajectories into daily-like segments, enriches segment tokens with precomputed semantic embeddings from prompts, and processes them with a frozen LLM backbone to produce accurate location predictions while reducing computational load. Key contributions include temporal tokenization, prompt-guided semantic context integration, and a parameter-efficient frozen-backbone design that yields improvements in accuracy (notably 2.4% overall and 5.0% on weekends) and training speed (about 24.6% faster). The approach demonstrates strong performance across three real-world mobility datasets and offers scalable deployment with various pretrained LLMs, highlighting practical impact for large-scale mobility forecasting.

Abstract

Predicting human mobility is inherently challenging due to complex long-range dependencies and multi-scale periodic behaviors. To address this, we introduce RHYTHM (Reasoning with Hierarchical Temporal Tokenization for Human Mobility), a unified framework that leverages large language models (LLMs) as general-purpose spatio-temporal predictors and trajectory reasoners. Methodologically, RHYTHM employs temporal tokenization to partition each trajectory into daily segments and encode them as discrete tokens with hierarchical attention that captures both daily and weekly dependencies, thereby quadratically reducing the sequence length while preserving cyclical information. Additionally, we enrich token representations by adding pre-computed prompt embeddings for trajectory segments and prediction targets via a frozen LLM, and feeding these combined embeddings back into the LLM backbone to capture complex interdependencies. Computationally, RHYTHM keeps the pretrained LLM backbone frozen, yielding faster training and lower memory usage. We evaluate our model against state-of-the-art methods using three real-world datasets. Notably, RHYTHM achieves a 2.4% improvement in overall accuracy, a 5.0% increase on weekends, and a 24.6% reduction in training time. Code is publicly available at https://github.com/he-h/rhythm.

Figures (5)

• Figure 1: Motivation for RHYTHM. Instead of processing entire trajectories as a continuous sequence, RHYTHM segments them into tokens to better capture periodic patterns.
• Figure 2: The proposed architecture of RHYTHM. Our framework processes historical trajectories through spatio-temporal embedding and temporal tokenization (b), capturing local and global dependencies via hierarchical attention. Segment representations are enriched with semantic embeddings from trajectory information, while future timestamps incorporate task description context (a). This combined sequence passes through a frozen LLM backbone with output projection to generate location predictions.
• Figure 3: Training Speed vs. performance of RHYTHM and baseline models on the Sapporo Dataset.
• Figure 4: Efficiency comparison of alternative LLMs, evaluated by the same configuration of \ref{['tab:scale']}.
• Figure 5: Weekly (left) and daily (right) accuracy trends of RHYTHM and baselines on Sapporo. These results illustrate that prediction performance fluctuates over both daily and weekly intervals.