Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CC-GPX: Extracting High-Quality Annotated Geospatial Data from Common Crawl

Ilya Ilyankou, Meihui Wang, Stefano Cavazzi, James Haworth

TL;DR

This work introduces a scalable pipeline to extract annotated geospatial data from Common Crawl by identifying GPX tracks, filtering for quality, masking personal information, translating descriptions, and augmenting elevation data. The resulting 1,416 track–description pairs across multiple languages provide a multimodal dataset suitable for trajectory generation, language-conditioned description models, and GeoAI fine-tuning. Key contributions include a reproducible data collection and processing workflow, careful privacy-preserving steps, and a multilingual, geographically diverse dataset derived from CC. The approach demonstrates the practicality of leveraging CC for geospatial resources and lays groundwork for expanding annotated geospatial data to tens of thousands of samples across historical CC releases.

Abstract

The Common Crawl (CC) corpus is the largest open web crawl dataset containing 9.5+ petabytes of data captured since 2008. The dataset is instrumental in training large language models, and as such it has been studied for (un)desirable content, and distilled for smaller, domain-specific datasets. However, to our knowledge, no research has been dedicated to using CC as a source of annotated geospatial data. In this paper, we introduce an efficient pipeline to extract annotated user-generated tracks from GPX files found in CC, and the resulting multimodal dataset with 1,416 pairings of human-written descriptions and MultiLineString vector data from the 6 most recent CC releases. The dataset can be used to study people's outdoor activity patterns, the way people talk about their outdoor experiences, as well as for developing trajectory generation or track annotation models, or for various other problems in place of synthetically generated routes. Our reproducible code is available on GitHub: https://github.com/ilyankou/cc-gpx

CC-GPX: Extracting High-Quality Annotated Geospatial Data from Common Crawl

TL;DR

This work introduces a scalable pipeline to extract annotated geospatial data from Common Crawl by identifying GPX tracks, filtering for quality, masking personal information, translating descriptions, and augmenting elevation data. The resulting 1,416 track–description pairs across multiple languages provide a multimodal dataset suitable for trajectory generation, language-conditioned description models, and GeoAI fine-tuning. Key contributions include a reproducible data collection and processing workflow, careful privacy-preserving steps, and a multilingual, geographically diverse dataset derived from CC. The approach demonstrates the practicality of leveraging CC for geospatial resources and lays groundwork for expanding annotated geospatial data to tens of thousands of samples across historical CC releases.

Abstract

The Common Crawl (CC) corpus is the largest open web crawl dataset containing 9.5+ petabytes of data captured since 2008. The dataset is instrumental in training large language models, and as such it has been studied for (un)desirable content, and distilled for smaller, domain-specific datasets. However, to our knowledge, no research has been dedicated to using CC as a source of annotated geospatial data. In this paper, we introduce an efficient pipeline to extract annotated user-generated tracks from GPX files found in CC, and the resulting multimodal dataset with 1,416 pairings of human-written descriptions and MultiLineString vector data from the 6 most recent CC releases. The dataset can be used to study people's outdoor activity patterns, the way people talk about their outdoor experiences, as well as for developing trajectory generation or track annotation models, or for various other problems in place of synthetically generated routes. Our reproducible code is available on GitHub: https://github.com/ilyankou/cc-gpx
Paper Structure (10 sections, 3 figures, 1 table)

This paper contains 10 sections, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Data Collection and Processing
  3. Downloading GPX files from CC
  4. Identifying high-quality tracks
  5. Identifying activity descriptions
  6. Removing personal information
  7. Translating descriptions to English
  8. Adding missing elevation
  9. Dataset & Potential Applications
  10. Conclusion

Figures (3)

  • Figure 1: A 18.2 km (11.3 mi) route in the UK. The description reads: 'Longbridge to Whalley Slowway following part of the Ribble Way. Difficult to find a good crossing of the A59. The crossing chosen crosses the road from footpath to footpath in a place with good visibility. The road junctions/bridges were actually worse as would need to walk along a fast road with no pavement rather than just cross once at right angles. This crossing sets up good sections without roads. Good spacing of waypoints at Old Langho and Ribchester.'
  • Figure 2: A 13.8 km (8.6 mi) circular route in Germany. The description in German reads: 'Der Weg ist sehr gut gekennzeichnet mit einem schwarzen Hirschkäfer (Hootzemann) auf weißem Grund. Mein Start- und Zielpunkt war das Schützenhaus Eiweiler in der Nähe der Großwald-Brauerei.' The English translation is: 'The path is very well marked with a black deer beetle (Hootzemann) on white ground. My starting and finishing point was the Schützenhaus Eiweiler near the Großwald brewery.'
  • Figure 3: Select dataset properties.