Geolocation Representation from Large Language Models are Generic Enhancers for Spatio-Temporal Learning

TL;DR

This work tackles the challenge of obtaining universal geolocation representations by proposing LLMGeovec, a training-free pipeline that derives coordinates embeddings from pre-trained LLMs via OpenStreetMap prompts. By two-phase construction—prompt generation and text embedding—the method yields global coverage and serves as a plug-in spatial prior for GP, LTSF, and GSTF models, demonstrated through lightweight adapters and feature concatenation. Empirical results show substantial improvements across multi-scale GP tasks and across diverse LTSF and GSTF benchmarks, with zero-shot transfer illustrating broad generalizability and potential to reduce reliance on expensive input data. The approach offers a scalable, data-efficient pathway to infuse rich geographic semantics into spatio-temporal learning, with promising directions toward larger LLMs and deeper integration into pre-trained foundational models.

Abstract

In the geospatial domain, universal representation models are significantly less prevalent than their extensive use in natural language processing and computer vision. This discrepancy arises primarily from the high costs associated with the input of existing representation models, which often require street views and mobility data. To address this, we develop a novel, training-free method that leverages large language models (LLMs) and auxiliary map data from OpenStreetMap to derive geolocation representations (LLMGeovec). LLMGeovec can represent the geographic semantics of city, country, and global scales, which acts as a generic enhancer for spatio-temporal learning. Specifically, by direct feature concatenation, we introduce a simple yet effective paradigm for enhancing multiple spatio-temporal tasks including geographic prediction (GP), long-term time series forecasting (LTSF), and graph-based spatio-temporal forecasting (GSTF). LLMGeovec can seamlessly integrate into a wide spectrum of spatio-temporal learning models, providing immediate enhancements. Experimental results demonstrate that LLMGeovec achieves global coverage and significantly boosts the performance of leading GP, LTSF, and GSTF models. Our codes are available at \url{https://github.com/Umaruchain/LLMGeovec}.

Figures (2)

• Figure 1: Our geolocation representation method consists of two phases: firstly, generating geolocation prompts for coordinates from map data, and then generating representations for text descriptions from pre-trained LLMs. It achieves global coverage and generates representations that can be used in various spatio-temporal tasks.
• Figure 2: Comparsion in zero-shot transfer scenarios.