Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Geospatial Mechanistic Interpretability of Large Language Models

Stef De Sabbata, Stefano Mizzaro, Kevin Roitero

TL;DR

The chapter addresses how Large Language Models encode geographic information and spatial reasoning, proposing a geospatial mechanistic interpretability framework that combines probing, mechanistic interpretability, and spatial analysis. Using placename prompts and a 7B-scale model, it analyzes internal activations across layers with Moran's $I$ statistics and applies a sparse autoencoder to disentangle geospatial features, revealing that many geographic signals are polysemantic and sparsely distributed, while a subset of features shows clear spatial structure. The findings support the superposition hypothesis and demonstrate that sparse, interpretable geospatial directions can be extracted, albeit only a minority of features exhibit strong spatial coherence ($p<.01$, $I\ge 0.3$). This work offers a path toward interpretable geospatial representations in foundation models, with implications for geography-focused AI safety, bias analysis, and GIS-like applications, and suggests avenues for integrating external geographic knowledge and multilingual data. $\,$

Abstract

Large Language Models (LLMs) have demonstrated unprecedented capabilities across various natural language processing tasks. Their ability to process and generate viable text and code has made them ubiquitous in many fields, while their deployment as knowledge bases and "reasoning" tools remains an area of ongoing research. In geography, a growing body of literature has been focusing on evaluating LLMs' geographical knowledge and their ability to perform spatial reasoning. However, very little is still known about the internal functioning of these models, especially about how they process geographical information. In this chapter, we establish a novel framework for the study of geospatial mechanistic interpretability - using spatial analysis to reverse engineer how LLMs handle geographical information. Our aim is to advance our understanding of the internal representations that these complex models generate while processing geographical information - what one might call "how LLMs think about geographic information" if such phrasing was not an undue anthropomorphism. We first outline the use of probing in revealing internal structures within LLMs. We then introduce the field of mechanistic interpretability, discussing the superposition hypothesis and the role of sparse autoencoders in disentangling polysemantic internal representations of LLMs into more interpretable, monosemantic features. In our experiments, we use spatial autocorrelation to show how features obtained for placenames display spatial patterns related to their geographic location and can thus be interpreted geospatially, providing insights into how these models process geographical information. We conclude by discussing how our framework can help shape the study and use of foundation models in geography.

Geospatial Mechanistic Interpretability of Large Language Models

TL;DR

The chapter addresses how Large Language Models encode geographic information and spatial reasoning, proposing a geospatial mechanistic interpretability framework that combines probing, mechanistic interpretability, and spatial analysis. Using placename prompts and a 7B-scale model, it analyzes internal activations across layers with Moran's statistics and applies a sparse autoencoder to disentangle geospatial features, revealing that many geographic signals are polysemantic and sparsely distributed, while a subset of features shows clear spatial structure. The findings support the superposition hypothesis and demonstrate that sparse, interpretable geospatial directions can be extracted, albeit only a minority of features exhibit strong spatial coherence (, ). This work offers a path toward interpretable geospatial representations in foundation models, with implications for geography-focused AI safety, bias analysis, and GIS-like applications, and suggests avenues for integrating external geographic knowledge and multilingual data.

Abstract

Large Language Models (LLMs) have demonstrated unprecedented capabilities across various natural language processing tasks. Their ability to process and generate viable text and code has made them ubiquitous in many fields, while their deployment as knowledge bases and "reasoning" tools remains an area of ongoing research. In geography, a growing body of literature has been focusing on evaluating LLMs' geographical knowledge and their ability to perform spatial reasoning. However, very little is still known about the internal functioning of these models, especially about how they process geographical information. In this chapter, we establish a novel framework for the study of geospatial mechanistic interpretability - using spatial analysis to reverse engineer how LLMs handle geographical information. Our aim is to advance our understanding of the internal representations that these complex models generate while processing geographical information - what one might call "how LLMs think about geographic information" if such phrasing was not an undue anthropomorphism. We first outline the use of probing in revealing internal structures within LLMs. We then introduce the field of mechanistic interpretability, discussing the superposition hypothesis and the role of sparse autoencoders in disentangling polysemantic internal representations of LLMs into more interpretable, monosemantic features. In our experiments, we use spatial autocorrelation to show how features obtained for placenames display spatial patterns related to their geographic location and can thus be interpreted geospatially, providing insights into how these models process geographical information. We conclude by discussing how our framework can help shape the study and use of foundation models in geography.
Paper Structure (13 sections, 3 figures)

This paper contains 13 sections, 3 figures.

Table of Contents

  1. Introduction
  2. Probing
  3. Introduction to probing
  4. Probing for geographical information
  5. Spatial analysis of internal representations
  6. Methodology and data
  7. Results
  8. Mechanistic interpretability
  9. Introduction to mechanistic interpretability
  10. Mechanistic interpretability via sparse autoencoders and spatial analysis
  11. Methodology and data
  12. Results
  13. Discussion and Conclusions

Figures (3)

  • Figure 1: An example illustrating the extraction of the activations from an LLM (top); the use of the activations in a linear probe to predict the latitude and longitude of the place mentioned in the input (bottom-left) and a sparse autoencoder (bottom-right); and the use of spatial autocorrelation to analyse the activations and the sparse features (centre). Our approach encompasses the latter two components.
  • Figure 2: Activations captured for the input placenames at different layers of the LLM (left for each region) and their local spatial autocorrelation (local Moran's $I$ clusters, $p<.01$, right for each region), illustrating the polysemantic nature of its internal representations. Two neurons at layers 7 (a) and 15 (b) show high values for the State of New York and Northern Ireland and very low values for northern Italy. A neuron at layer (c) 15 shows high values for several UK cities. A neuron at layer 31 (d) shows high values for the State of New York and Northern Ireland and diverse values for provinces in Italy.
  • Figure 3: Features extracted from layer 15 through a sparse autoencoder (left for each region) and their local spatial autocorrelation (local Moran's $I$ clusters, $p<.01$, right for each region): (a) Wales as a region part of prompt; (b) south of Italy as a region activating a seemingly monosemantic feature; (c) north-east of Italy and north-west of England as regions activating a seemingly polysemantic feature; and (d) a representation of "city" highlighting New York City and London, amongst others.