Classical Feature Embeddings Help in BERT-Based Human Mobility Prediction

TL;DR

This work tackles the challenge of predicting human mobility by leveraging richer semantic context than traditional location-only or time-as-auxiliary-input models. It introduces STaBERT, a Semantic-Temporal aware BERT that fuses POI embeddings and derived temporal descriptors into a unified representation built on a Transformer backbone (LP-BERT). The approach yields substantial improvements in GEO-BLEU for both single-city and multi-city predictions, demonstrating the value of jointly modeling POI semantics and temporal dynamics. The results indicate STaBERT is an efficient, scalable solution with practical impact for disaster relief, city planning, and public health, while remaining adaptable to alternative spatial descriptors when POI data are unavailable.

Abstract

Human mobility forecasting is crucial for disaster relief, city planning, and public health. However, existing models either only model location sequences or include time information merely as auxiliary input, thereby failing to leverage the rich semantic context provided by points of interest (POIs). To address this, we enrich a BERT-based mobility model with derived temporal descriptors and POI embeddings to better capture the semantics underlying human movement. We propose STaBERT (Semantic-Temporal aware BERT), which integrates both POI and temporal information at each location to construct a unified, semantically enriched representation of mobility. Experimental results show that STaBERT significantly improves prediction accuracy: for single-city prediction, the GEO-BLEU score improved from 0.34 to 0.75; for multi-city prediction, from 0.34 to 0.56.

Figures (2)

• Figure 1: STaBERT model structure. Enhancements highlighted in orange
• Figure 2: User 80004 Trajectory