The Sonar Moment: Benchmarking Audio-Language Models in Audio Geo-Localization

TL;DR

AGL1K introduces the first audio geo-localization benchmark for audio-language models, combining a crowd-sourced, geo-tagged audio corpus with a principled Audio Localizability metric to filter highly informative samples. Across 1,444 clips from 72 countries, the study reveals emergent geo-localization capabilities in ALMs, with a pronounced advantage for closed-source models and language-driven reasoning as a dominant cue. The authors dissect model reasoning, regional biases, and error modes, highlighting the need for improved fine-grained perception and robust, multi-clue fusion to achieve reliable geospatial localization. This benchmark provides a pathway to advance ALMs’ geospatial reasoning and has implications for public-safety and misinformation-context tasks. The work also offers an interactive platform and clear evaluation protocols to foster further research in audio-based geographic inference.

Abstract

Geo-localization aims to infer the geographic origin of a given signal. In computer vision, geo-localization has served as a demanding benchmark for compositional reasoning and is relevant to public safety. In contrast, progress on audio geo-localization has been constrained by the lack of high-quality audio-location pairs. To address this gap, we introduce AGL1K, the first audio geo-localization benchmark for audio language models (ALMs), spanning 72 countries and territories. To extract reliably localizable samples from a crowd-sourced platform, we propose the Audio Localizability metric that quantifies the informativeness of each recording, yielding 1,444 curated audio clips. Evaluations on 16 ALMs show that ALMs have emerged with audio geo-localization capability. We find that closed-source models substantially outperform open-source models, and that linguistic clues often dominate as a scaffold for prediction. We further analyze ALMs' reasoning traces, regional bias, error causes, and the interpretability of the localizability metric. Overall, AGL1K establishes a benchmark for audio geo-localization and may advance ALMs with better geospatial reasoning capability.

Figures (10)

• Figure 2: Overview of the benchmark construction framework. AGL1K is curated from the crowd-sourced Aporee platform. The recordings are first filtered using four acoustic filters, followed by our proposed Audio Localizability measure, which quantifies the geo-informativeness of each sample.
• Figure 3: Top Positive and Negative Categories.
• Figure 4: Localizability Examples. Localizability increases from right to left.
• Figure 5: Benchmark examples. We select three representative audio samples and present the distribution of their audio clues, the reasoning process of Gemini 3 Pro, and predictions of three other ALMs.
• Figure 6: The continent-level prediction inequality in audio geo-localization. The i-j entry indicates that the truth is the portion of the continent in the i-th line that is predicted to be continent in the j-th row.