Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SoundPlot: An Open-Source Framework for Birdsong Acoustic Analysis and Neural Synthesis with Interactive 3D Visualization

Naqcho Ali Mehdi, Mohammad Adeel, Aizaz Ali Larik

TL;DR

The system transforms audio signals into a multi-dimensional acoustic feature space, enabling real-time visualization of temporal dynamics in 3D using web-based interactive graphics, and demonstrates the framework's capabilities through comprehensive waveform analysis, spectrogram comparisons, and feature space evaluation using Principal Component Analysis (PCA).

Abstract

We present SoundPlot, an open-source framework for analyzing avian vocalizations through acoustic feature extraction, dimensionality reduction, and neural audio synthesis. The system transforms audio signals into a multi-dimensional acoustic feature space, enabling real-time visualization of temporal dynamics in 3D using web-based interactive graphics. Our framework implements a complete analysis-synthesis pipeline that extracts spectral features (centroid, bandwidth, contrast), pitch contours via probabilistic YIN (pYIN), and mel-frequency cepstral coefficients (MFCCs), mapping them to a unified timbre space for visualization. Audio reconstruction employs the Griffin-Lim phase estimation algorithm applied to mel spectrograms. The accompanying Three.js-based interface provides dual-viewport visualization comparing original and synthesized audio trajectories with independent playback controls. We demonstrate the framework's capabilities through comprehensive waveform analysis, spectrogram comparisons, and feature space evaluation using Principal Component Analysis (PCA). Quantitative evaluation shows mel spectrogram correlation scores exceeding 0.92, indicating high-fidelity preservation of perceptual acoustic structure. SoundPlot is released under the MIT License to facilitate research in bioacoustics, audio signal processing, and computational ethology.

SoundPlot: An Open-Source Framework for Birdsong Acoustic Analysis and Neural Synthesis with Interactive 3D Visualization

TL;DR

The system transforms audio signals into a multi-dimensional acoustic feature space, enabling real-time visualization of temporal dynamics in 3D using web-based interactive graphics, and demonstrates the framework's capabilities through comprehensive waveform analysis, spectrogram comparisons, and feature space evaluation using Principal Component Analysis (PCA).

Abstract

We present SoundPlot, an open-source framework for analyzing avian vocalizations through acoustic feature extraction, dimensionality reduction, and neural audio synthesis. The system transforms audio signals into a multi-dimensional acoustic feature space, enabling real-time visualization of temporal dynamics in 3D using web-based interactive graphics. Our framework implements a complete analysis-synthesis pipeline that extracts spectral features (centroid, bandwidth, contrast), pitch contours via probabilistic YIN (pYIN), and mel-frequency cepstral coefficients (MFCCs), mapping them to a unified timbre space for visualization. Audio reconstruction employs the Griffin-Lim phase estimation algorithm applied to mel spectrograms. The accompanying Three.js-based interface provides dual-viewport visualization comparing original and synthesized audio trajectories with independent playback controls. We demonstrate the framework's capabilities through comprehensive waveform analysis, spectrogram comparisons, and feature space evaluation using Principal Component Analysis (PCA). Quantitative evaluation shows mel spectrogram correlation scores exceeding 0.92, indicating high-fidelity preservation of perceptual acoustic structure. SoundPlot is released under the MIT License to facilitate research in bioacoustics, audio signal processing, and computational ethology.
Paper Structure (28 sections, 11 equations, 3 figures, 1 table)

This paper contains 28 sections, 11 equations, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Motivation
  3. Contributions
  4. Related Work
  5. Birdsong Analysis Systems
  6. Audio Feature Extraction
  7. Audio Synthesis and Phase Reconstruction
  8. Dimensionality Reduction and Visualization
  9. System Architecture
  10. Project Organization
  11. Audio Processing Pipeline
  12. Methodology
  13. Feature Extraction Mathematics
  14. Mel Spectrogram Synthesis
  15. Quality Metrics
  16. ...and 13 more sections

Figures (3)

  • Figure 1: SoundPlot project directory structure showing the modular organization of audio processing, feature extraction, analysis, and visualization components.
  • Figure 2: Comprehensive comparison of original and synthesized birdsong. Top: temporal waveforms; Middle: STFT spectrograms; Bottom: mel spectrograms with perceptual frequency scaling. Quantitative metrics: SNR = -0.81 dB, Spectral Corr. = 0.566, Mel Corr. = 0.929.
  • Figure 3: Feature space comparison via Principal Component Analysis (PCA). Blue points: original audio; Green points: synthesized audio; Gray lines: temporal drift between corresponding frames. The preserved structure indicates successful capture of temporal dynamics despite the synthesis transformation.