Are Video Generation Models Geographically Fair? An Attraction-Centric Evaluation of Global Visual Knowledge

TL;DR

This paper investigates whether text-to-video models encode geographically equitable visual knowledge by introducing GAP, a Geo-Attraction Landmark Probing framework. It constructs GeoAttraction-500, a benchmark of 500 globally distributed attractions, and pairs it with a multi-metric evaluation that separates overall video quality from attraction-specific knowledge using Patch-Level CLIP, Keypoint-Based Local Alignment, and VLM-based judgments, validated against human evaluation. Applying GAP to Sora 2, the study finds relatively uniform geo-knowledge across regions and popularity levels, with only weak dependence on attraction popularity, suggesting robust global visual knowledge in current video generators. The work provides a principled, scalable approach for geo-reliability assessment in video generation and highlights the need for ongoing, diverse evaluations as models evolve.

Abstract

Recent advances in text-to-video generation have produced visually compelling results, yet it remains unclear whether these models encode geographically equitable visual knowledge. In this work, we investigate the geo-equity and geographically grounded visual knowledge of text-to-video models through an attraction-centric evaluation. We introduce Geo-Attraction Landmark Probing (GAP), a systematic framework for assessing how faithfully models synthesize tourist attractions from diverse regions, and construct GEOATTRACTION-500, a benchmark of 500 globally distributed attractions spanning varied regions and popularity levels. GAP integrates complementary metrics that disentangle overall video quality from attraction-specific knowledge, including global structural alignment, fine-grained keypoint-based alignment, and vision-language model judgments, all validated against human evaluation. Applying GAP to the state-of-the-art text-to-video model Sora 2, we find that, contrary to common assumptions of strong geographic bias, the model exhibits a relatively uniform level of geographically grounded visual knowledge across regions, development levels, and cultural groupings, with only weak dependence on attraction popularity. These results suggest that current text-to-video models express global visual knowledge more evenly than expected, highlighting both their promise for globally deployed applications and the need for continued evaluation as such systems evolve.

Figures (4)

• Figure 1: Overview of the evaluation metrics used in Geo-Attraction Landmark Probing (GAP). GAP integrates complementary knowledge-oriented and quality-oriented metrics to assess generated videos. Patch-Level CLIP measures global structural alignment via patch-wise similarity, Keypoint-Based Local Alignment evaluates fine-grained geometric and textural consistency, and VLM-as-A-Judge provides semantic assessments of global and local alignment. In contrast, AIGVE-MACS evaluates overall video quality independently of attraction-specific knowledge.
• Figure 2: Dataset statistics. Top: Geographic distribution of attractions by country, showing broad global coverage. Bottom left: Distribution of pageviews of attraction landmarks in log scale. Bottom right: Top 10 hierarchical attraction landmark categories, dominated by architectural and cultural structures.
• Figure 3: Spearman rank correlation coefficient between human evaluation scores and automatic metrics.
• Figure 4: Qualitative case studies illustrating model performance across attractions with varying popularity. For each attraction, the top image shows the ground-truth reference, while the bottom image is generated by the video generation model. Orange dashed outlines highlight regions where the generated content deviates from the reference, indicating structural or geometric errors.