Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

TimeSpot: Benchmarking Geo-Temporal Understanding in Vision-Language Models in Real-World Settings

Azmine Toushik Wasi, Shahriyar Zaman Ridoy, Koushik Ahamed Tonmoy, Kinga Tshering, S. M. Muhtasimul Hasan, Wahid Faisal, Tasnim Mohiuddin, Md Rizwan Parvez

TL;DR

TimeSpot is introduced, a benchmark for evaluating real-world geo-temporal reasoning in VLMs and requires structured prediction of temporal attributes and geographic attributes directly from visual evidence.

Abstract

Geo-temporal understanding, the ability to infer location, time, and contextual properties from visual input alone, underpins applications such as disaster management, traffic planning, embodied navigation, world modeling, and geography education. Although recent vision-language models (VLMs) have advanced image geo-localization using cues like landmarks and road signs, their ability to reason about temporal signals and physically grounded spatial cues remains limited. To address this gap, we introduce TimeSpot, a benchmark for evaluating real-world geo-temporal reasoning in VLMs. TimeSpot comprises 1,455 ground-level images from 80 countries and requires structured prediction of temporal attributes (season, month, time of day, daylight phase) and geographic attributes (continent, country, climate zone, environment type, latitude-longitude) directly from visual evidence. It also includes spatial-temporal reasoning tasks that test physical plausibility under real-world uncertainty. Evaluations of state-of-the-art open- and closed-source VLMs show low performance, particularly for temporal inference. While supervised fine-tuning yields improvements, results remain insufficient, highlighting the need for new methods to achieve robust, physically grounded geo-temporal understanding. TimeSpot is available at: https://TimeSpot-GT.github.io.

TimeSpot: Benchmarking Geo-Temporal Understanding in Vision-Language Models in Real-World Settings

TL;DR

TimeSpot is introduced, a benchmark for evaluating real-world geo-temporal reasoning in VLMs and requires structured prediction of temporal attributes and geographic attributes directly from visual evidence.

Abstract

Geo-temporal understanding, the ability to infer location, time, and contextual properties from visual input alone, underpins applications such as disaster management, traffic planning, embodied navigation, world modeling, and geography education. Although recent vision-language models (VLMs) have advanced image geo-localization using cues like landmarks and road signs, their ability to reason about temporal signals and physically grounded spatial cues remains limited. To address this gap, we introduce TimeSpot, a benchmark for evaluating real-world geo-temporal reasoning in VLMs. TimeSpot comprises 1,455 ground-level images from 80 countries and requires structured prediction of temporal attributes (season, month, time of day, daylight phase) and geographic attributes (continent, country, climate zone, environment type, latitude-longitude) directly from visual evidence. It also includes spatial-temporal reasoning tasks that test physical plausibility under real-world uncertainty. Evaluations of state-of-the-art open- and closed-source VLMs show low performance, particularly for temporal inference. While supervised fine-tuning yields improvements, results remain insufficient, highlighting the need for new methods to achieve robust, physically grounded geo-temporal understanding. TimeSpot is available at: https://TimeSpot-GT.github.io.
Paper Structure (56 sections, 8 equations, 33 figures, 20 tables)

This paper contains 56 sections, 8 equations, 33 figures, 20 tables.

Table of Contents

  1. Introduction
  2. Preliminaries and Related Work
  3. TimeSpot Benchmark
  4. Benchmark Construction
  5. Experiments and Evaluation
  6. Results and Analysis
  7. Performance Analysis
  8. Error Analysis
  9. Dataset Stability
  10. Performance Improvement with SFT
  11. Concluding Remarks
  12. Extended Related Work
  13. Remote sensing and aerial geospatial VLMs.
  14. Ground-level localization and cross-view benchmarks.
  15. Image-based geolocation with large VLMs.
  16. ...and 41 more sections

Figures (33)

  • Figure 1: TimeSpot development and evaluation pipeline, contrasting VLM predictions with expert-annotated ground truth for geo-temporal accuracy and physical consistency.
  • Figure 2: Global coverage of TimeSpot. Locations of all 1,455 ground, level photos across 80 countries. Each marker denotes an image coordinate; colors indicate country (top contributors listed; “Others” aggregates the remainder).
  • Figure 3: Qualitative results on TimeSpot (Gemini-2.5-Flash). Each panel pairs the image with ground truth and model outputs, country, daylight phase, and local time, along with MD and $|\Delta t|$.
  • Figure 4: SFT performance trends over epochs.
  • Figure 5: Illustration of the TimeSpot benchmark for geo-temporal understanding. Models must infer temporal attributes (season, month, time of day, daylight phase) and geographic attributes (continent, country, climate zone, environment type, coordinates) directly from visual input. Left: example images. Center: distributions of primary temporal cues (e.g., architecture, natural biome, topography) and geolocation cues (e.g., sun/shadows, vegetation, snow/ice). Right: an example prediction, highlighting the integration of diverse and subtle cues required for reliable reasoning.
  • ...and 28 more figures