Geo-ATBench: A Benchmark for Geospatial Audio Tagging with Geospatial Semantic Context

TL;DR

The Geo-AT task, benchmark Geo-ATBench, and reproducible geo-audio fusion framework GeoFusion-AT provide a foundation for studying AT with geospatial semantic context within the CASA community.

Abstract

Environmental sound understanding in computational auditory scene analysis (CASA) is often formulated as an audio-only recognition problem. This formulation leaves a persistent drawback in multi-label audio tagging (AT): acoustic similarity can make certain events difficult to separate from waveforms alone. In such cases, disambiguating cues often lie outside the waveform. Geospatial semantic context (GSC), derived from geographic information system data, e.g., points of interest (POI), provides location-tied environmental priors that can help reduce this ambiguity. A systematic study of this direction is enabled through the proposed geospatial audio tagging (Geo-AT) task, which conditions multi-label sound event tagging on GSC alongside audio. To benchmark Geo-AT, Geo-ATBench is introduced as a polyphonic audio benchmark with geographical annotations, containing 10.71 hours of audio across 28 event categories; each clip is paired with a GSC representation from 11 semantic context categories. GeoFusion-AT is proposed as a unified geo-audio fusion framework that evaluates feature-, representation-, and decision-level fusion on representative audio backbones, with audio- and GSC-only baselines. Results show that incorporating GSC improves AT performance, especially on acoustically confounded labels, indicating geospatial semantics provide effective priors beyond audio alone. A crowdsourced listening study with 10 participants on 579 samples shows that there is no significant difference in performance between models on Geo-ATBench labels and aggregated human labels, supporting Geo-ATBench as a human-aligned benchmark. The Geo-AT task, benchmark Geo-ATBench, and reproducible geo-audio fusion framework GeoFusion-AT provide a foundation for studying AT with geospatial semantic context within the CASA community. Dataset, code, models are on homepage (https://github.com/WuYanru2002/Geo-ATBench).

Figures (12)

• Figure 1: The number of recordings with GPS information uploaded to Freesound each year.
• Figure 2: Summary of sound classes and acoustic similarity. (Left) Distribution of three coarse-grained sound classes. (Right) Intra-class similarity across 28 sound event classes computed from log-Mel spectrogram features.
• Figure 3: Sankey diagram summarizing co-occurrence links from left to right: 3 coarse-grained sound classes, 28 fine-grained sound event classes, GSC types, and the Geo-ATBench dataset. Flow width indicates co-occurrence strength. This diagram represents the distribution of audio events and GSC types within the dataset, and is not intended to imply precise real-world relationships, as sound occurrences can vary significantly depending on the specific geographical context (e.g., residential roads vs highways).
• Figure 4: Overall architecture of the GeoFusion-AT framework for Geo-AT task.
• Figure 5: Instantiations of GeoFusion-Early (feature-level fusion).