Histogram-based Parameter-efficient Tuning for Passive Sonar Classification
Amirmohammad Mohammadi, Davelle Carreiro, Alexandra Van Dine, Joshua Peeples
TL;DR
The paper addresses the challenge of distribution shifts in passive sonar data when applying parameter-efficient transfer learning. It introduces histogram-based parameter-efficient tuning (HPT), which injects a 1D histogram layer into a transformer to summarize intermediate feature distributions and modulate embeddings, complementing the self-attention pathway. Empirical results on ShipsEar, DeepShip, and VTUAD show HPT often outperforms conventional adapters, provides faster convergence, and yields representations closer to fully fine-tuned models, with notable gains on VTUAD in non-shared settings (e.g., 91.8% accuracy with 153.0K trainable params). The work demonstrates a scalable, distribution-aware PETL approach suitable for resource-constrained environments and points to future directions such as adaptive binning and hybridization with other PETL techniques.
Abstract
Parameter-efficient transfer learning (PETL) methods adapt large artificial neural networks to downstream tasks without fine-tuning the entire model. However, existing additive methods, such as adapters, sometimes struggle to capture distributional shifts in intermediate feature embeddings. We propose a novel histogram-based parameter-efficient tuning (HPT) technique that captures the statistics of the target domain and modulates the embeddings. Experimental results on three downstream passive sonar datasets (ShipsEar, DeepShip, VTUAD) demonstrate that HPT outperforms conventional adapters. Notably, HPT achieves 91.8% vs. 89.8% accuracy on VTUAD. Furthermore, HPT trains faster and yields feature representations closer to those of fully fine-tuned models. Overall, HPT balances parameter savings and performance, providing a distribution-aware alternative to existing adapters and shows a promising direction for scalable transfer learning in resource-constrained environments. The code is publicly available: https://github.com/Advanced-Vision-and-Learning-Lab/HLAST_DeepShip_ParameterEfficient.