Analytic Incremental Learning For Sound Source Localization With Imbalance Rectification

TL;DR

This work tackles sound source localization (SSL) under generalized class-incremental learning (GCIL), where intra-task long-tailed DoA distributions and inter-task skews cause catastrophic forgetting. It introduces GCC-PHAT-based data augmentation (GDA) to synthesize tail-class samples while preserving inter-microphone correlations, and Analytic Dynamic Imbalance Rectifier (ADIR) to provide task-adaptive, regularized updates without storing past exemplars. The method, SSL-GCIL, employs a frozen initial feature extractor with a task-adaptive classifier and a closed-form, Gini-guided regularization scheme that adapts to distribution shifts. On the SSLR benchmark, SSL-GCIL achieves state-of-the-art performance (ACC 89.0%, MAE 5.3°, BWT 1.6) and demonstrates robust resistance to forgetting across noise levels, with ablations confirming the complementary benefits of GDA and ADIR. These results suggest practical, privacy-preserving continual SSL in evolving real-world environments, with potential extensions to audio-visual modalities and online adaptation.

Abstract

Sound source localization (SSL) demonstrates remarkable results in controlled settings but struggles in real-world deployment due to dual imbalance challenges: intra-task imbalance arising from long-tailed direction-of-arrival (DoA) distributions, and inter-task imbalance induced by cross-task skews and overlaps. These often lead to catastrophic forgetting, significantly degrading the localization accuracy. To mitigate these issues, we propose a unified framework with two key innovations. Specifically, we design a GCC-PHAT-based data augmentation (GDA) method that leverages peak characteristics to alleviate intra-task distribution skews. We also propose an Analytic dynamic imbalance rectifier (ADIR) with task-adaption regularization, which enables analytic updates that adapt to inter-task dynamics. On the SSLR benchmark, our proposal achieves state-of-the-art (SoTA) results of 89.0% accuracy, 5.3° mean absolute error, and 1.6 backward transfer, demonstrating robustness to evolving imbalances without exemplar storage.

Figures (2)

• Figure 1: (a) Block Diagram of our proposed method and (b) GCC-PHAT-based Data Augmentation ($\bigoplus$ indicates addition).
• Figure 2: Comparison of performance curves between the baseline method and our proposed SSL-GCIL approach.