Adaptive Evidence Weighting for Audio-Spatiotemporal Fusion

TL;DR

FINCH addresses robustly fusing heterogeneous, approximately independent evidence sources in bioacoustic classification by learning a per-sample gating function that adaptively weighs a spatiotemporal prior on top of a frozen audio classifier. The method preserves an audio-only fallback, bounds contextual influence, and is trained in a staged, modular fashion to keep the audio backbone fixed while learning fusion parameters. Across large benchmarks (CBI and BirdSet), adaptive gating yields improvements over audio-only and fixed-weight fusion, achieving state-of-the-art performance on CBI and competitive results on BirdSet while maintaining interpretability and safety. The approach highlights the practical value of selective contextual integration for ecological monitoring and offers a general framework applicable to other domains with multiple, heterogeneous, approximately independent evidence sources.

Abstract

Many machine learning systems have access to multiple sources of evidence for the same prediction target, yet these sources often differ in reliability and informativeness across inputs. In bioacoustic classification, species identity may be inferred both from the acoustic signal and from spatiotemporal context such as location and season; while Bayesian inference motivates multiplicative evidence combination, in practice we typically only have access to discriminative predictors rather than calibrated generative models. We introduce \textbf{F}usion under \textbf{IN}dependent \textbf{C}onditional \textbf{H}ypotheses (\textbf{FINCH}), an adaptive log-linear evidence fusion framework that integrates a pre-trained audio classifier with a structured spatiotemporal predictor. FINCH learns a per-sample gating function that estimates the reliability of contextual information from uncertainty and informativeness statistics. The resulting fusion family \emph{contains} the audio-only classifier as a special case and explicitly bounds the influence of contextual evidence, yielding a risk-contained hypothesis class with an interpretable audio-only fallback. Across benchmarks, FINCH consistently outperforms fixed-weight fusion and audio-only baselines, improving robustness and error trade-offs even when contextual information is weak in isolation. We achieve state-of-the-art performance on CBI and competitive or improved performance on several subsets of BirdSet using a lightweight, interpretable, evidence-based approach. Code is available: \texttt{\href{https://anonymous.4open.science/r/birdnoise-85CD/README.md}{anonymous-repository}}

Figures (3)

• Figure 1: Haemorhous mexicanus (North American House Finch), the namesake of our model.
• Figure 2: FINCH: adaptive log-linear evidence fusion.(A) An audio encoder and linear head produce an acoustic posterior $p_\theta(y\mid x)$ from the input spectrogram. (B) A spatiotemporal model maps context $s$ (location and time) to a prior $p_\psi(y\mid s)$. (C) A gating network computes a nonnegative per-sample weight $\omega(x,s)$ from uncertainty and informativeness features, yielding unnormalized log-scores $\log \tilde{p}_\omega(y\mid x,s)=\log p_\theta(y\mid x)+\omega(x,s)\log p_\psi(y\mid s)$, which are normalized by a softmax over $y$ to obtain $p_\omega(y\mid x,s)$. When $\omega(x,s)=0$, the audio-only model is recovered. (D) Directed acyclic graph illustrating the assumption that $x$ and $s$ are conditionally independent given the label $y$, motivating multiplicative fusion.
• Figure 3: Motivating examples of adaptive fusion at test time.Top: The acoustic model assigns low probability to the true class (Grey-crowned Rosy Finch), ranking several acoustically similar species higher. The spatiotemporal model is highly confident and geographically specific, and the fused posterior correctly recovers the prior’s top prediction with a high gating weight. Bottom: Neither the acoustic model (top prediction: Yellow-bellied Flycatcher) nor the spatiotemporal prior (diffuse over multiple warblers) is correct in isolation. However, adaptive fusion suppresses the acoustically dominant but ecologically implausible class, yielding the correct posterior prediction (Black-throated Green Warbler). In both cases, the learned gating weight differs substantially across inputs, illustrating why fixed-weight fusion would fail to recover these outcomes.