Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Reconsidering utility: unveiling the limitations of synthetic mobility data generation algorithms in real-life scenarios

Alexandra Kapp, Helena Mihaljević

TL;DR

This study critically evaluates five state-of-the-art synthetic mobility data generators for fine-grained trips under differential privacy, using map matching to road networks and a privacy-friendly OSRM routing baseline as a realism benchmark. It introduces a practitioner-centric utility framework that includes trip lengths, street-level spatial distributions, road preferences, and intersection traffic flow, all anchored by map-matching. The findings show that AdaTrace and PrivTrace offer the most promise under DP, but overall current methods struggle to produce realistic, navigable trips with meaningful temporal or user-level attributes, and TrajGAIL and DP-Loc face substantial feasibility or utility issues. The results suggest a shift away from seeking full flexibility with privacy toward targeted applications and possibly aggregate data releases, while highlighting the need for road-aware generative mechanisms and more robust, scalable privacy-preserving approaches.

Abstract

In recent years, there has been a surge in the development of models for the generation of synthetic mobility data. These models aim to facilitate the sharing of data while safeguarding privacy, all while ensuring high utility and flexibility regarding potential applications. However, current utility evaluation methods fail to fully account for real-life requirements. We evaluate the utility of five state-of-the-art synthesis approaches, each with and without the incorporation of differential privacy (DP) guarantees, in terms of real-world applicability. Specifically, we focus on so-called trip data that encode fine granular urban movements such as GPS-tracked taxi rides. Such data prove particularly valuable for downstream tasks at the road network level. Thus, our initial step involves appropriately map matching the synthetic data and subsequently comparing the resulting trips with those generated by the routing algorithm implemented in OpenStreetMap, which serves as an efficient and privacy-friendly baseline. Out of the five evaluated models, one fails to produce data within reasonable computation time and another generates too many jumps to meet the requirements for map matching. The remaining three models succeed to a certain degree in maintaining spatial distribution, one even with DP guarantees. However, all models struggle to produce meaningful sequences of geo-locations with reasonable trip lengths and to model traffic flow at intersections accurately. It is important to note that trip data encompasses various relevant characteristics beyond spatial distribution, such as temporal information, all of which are discarded by these models. Consequently, our results imply that current synthesis models fall short in their promise of high utility and flexibility.

Reconsidering utility: unveiling the limitations of synthetic mobility data generation algorithms in real-life scenarios

TL;DR

This study critically evaluates five state-of-the-art synthetic mobility data generators for fine-grained trips under differential privacy, using map matching to road networks and a privacy-friendly OSRM routing baseline as a realism benchmark. It introduces a practitioner-centric utility framework that includes trip lengths, street-level spatial distributions, road preferences, and intersection traffic flow, all anchored by map-matching. The findings show that AdaTrace and PrivTrace offer the most promise under DP, but overall current methods struggle to produce realistic, navigable trips with meaningful temporal or user-level attributes, and TrajGAIL and DP-Loc face substantial feasibility or utility issues. The results suggest a shift away from seeking full flexibility with privacy toward targeted applications and possibly aggregate data releases, while highlighting the need for road-aware generative mechanisms and more robust, scalable privacy-preserving approaches.

Abstract

In recent years, there has been a surge in the development of models for the generation of synthetic mobility data. These models aim to facilitate the sharing of data while safeguarding privacy, all while ensuring high utility and flexibility regarding potential applications. However, current utility evaluation methods fail to fully account for real-life requirements. We evaluate the utility of five state-of-the-art synthesis approaches, each with and without the incorporation of differential privacy (DP) guarantees, in terms of real-world applicability. Specifically, we focus on so-called trip data that encode fine granular urban movements such as GPS-tracked taxi rides. Such data prove particularly valuable for downstream tasks at the road network level. Thus, our initial step involves appropriately map matching the synthetic data and subsequently comparing the resulting trips with those generated by the routing algorithm implemented in OpenStreetMap, which serves as an efficient and privacy-friendly baseline. Out of the five evaluated models, one fails to produce data within reasonable computation time and another generates too many jumps to meet the requirements for map matching. The remaining three models succeed to a certain degree in maintaining spatial distribution, one even with DP guarantees. However, all models struggle to produce meaningful sequences of geo-locations with reasonable trip lengths and to model traffic flow at intersections accurately. It is important to note that trip data encompasses various relevant characteristics beyond spatial distribution, such as temporal information, all of which are discarded by these models. Consequently, our results imply that current synthesis models fall short in their promise of high utility and flexibility.
Paper Structure (21 sections, 1 equation, 19 figures, 7 tables)

This paper contains 21 sections, 1 equation, 19 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Synthesis Algorithms
  3. Utility measurement
  4. Experimental setup
  5. Raw dataset
  6. Configurations for synthesis models
  7. Matching and routing implementation
  8. Road preference survey
  9. Traffic flow evaluation
  10. Results
  11. Matching ability
  12. Trip lengths
  13. Traffic volume
  14. Road preference
  15. Statistical similarity
  16. ...and 6 more sections

Figures (19)

  • Figure 1: Schematic illustration: preferred segments (green cells) are traversed by the matched trip (green line), but not its routed counterpart (purple line), and vice versa, they are considered avoided (purple cells).
  • Figure 2: Example for raw (left) and AdaTrace (right) trip in four variations.
  • Figure 3: Examples of survey questions. The preference score is overlaid on a map cutout alongside selected roads. Answers are given via radio buttons for each road.
  • Figure 4: raw
  • Figure 5: AdaTrace
  • ...and 14 more figures