Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Data-Centric Framework for Machine Listening Projects: Addressing Large-Scale Data Acquisition and Labeling through Active Learning

Javier Naranjo-Alcazar, Jordi Grau-Haro, Ruben Ribes-Serrano, Pedro Zuccarello

TL;DR

The paper tackles the challenge of large-scale, context-rich audio data labeling for Machine Listening under constrained resources. It presents a data-centric framework combining IoT recording nodes, a structured audio/labeling database, and an Active Learning pipeline anchored by MAL-MF with batch sampling on PANNs CNN14 embeddings. Key contributions include hardware design for robust data capture, comprehensive AL/database architectures, and a scalable labeling workflow validated in a Valencia port case that labeled 6540 ten-second clips with a small team. The work offers a practical blueprint for deploying commercial-ready, labeled audio datasets without crowdsourcing and provides a reusable template adaptable to varying resource profiles.

Abstract

Machine Listening focuses on developing technologies to extract relevant information from audio signals. A critical aspect of these projects is the acquisition and labeling of contextualized data, which is inherently complex and requires specific resources and strategies. Despite the availability of some audio datasets, many are unsuitable for commercial applications. The paper emphasizes the importance of Active Learning (AL) using expert labelers over crowdsourcing, which often lacks detailed insights into dataset structures. AL is an iterative process combining human labelers and AI models to optimize the labeling budget by intelligently selecting samples for human review. This approach addresses the challenge of handling large, constantly growing datasets that exceed available computational resources and memory. The paper presents a comprehensive data-centric framework for Machine Listening projects, detailing the configuration of recording nodes, database structure, and labeling budget optimization in resource-constrained scenarios. Applied to an industrial port in Valencia, Spain, the framework successfully labeled 6540 ten-second audio samples over five months with a small team, demonstrating its effectiveness and adaptability to various resource availability situations. Acknowledgments: The participation of Javier Naranjo-Alcazar, Jordi Grau-Haro and Pedro Zuccarello in this research was funded by the Valencian Institute for Business Competitiveness (IVACE) and the FEDER funds by means of project Soroll-IA2 (IMDEEA/2023/91). The research carried out for this publication has been partially funded by the project STARRING-NEURO (PID2022-137048OA-C44) funded by the Ministry of Science, Innovation and Universities of Spain and the European Union.

A Data-Centric Framework for Machine Listening Projects: Addressing Large-Scale Data Acquisition and Labeling through Active Learning

TL;DR

The paper tackles the challenge of large-scale, context-rich audio data labeling for Machine Listening under constrained resources. It presents a data-centric framework combining IoT recording nodes, a structured audio/labeling database, and an Active Learning pipeline anchored by MAL-MF with batch sampling on PANNs CNN14 embeddings. Key contributions include hardware design for robust data capture, comprehensive AL/database architectures, and a scalable labeling workflow validated in a Valencia port case that labeled 6540 ten-second clips with a small team. The work offers a practical blueprint for deploying commercial-ready, labeled audio datasets without crowdsourcing and provides a reusable template adaptable to varying resource profiles.

Abstract

Machine Listening focuses on developing technologies to extract relevant information from audio signals. A critical aspect of these projects is the acquisition and labeling of contextualized data, which is inherently complex and requires specific resources and strategies. Despite the availability of some audio datasets, many are unsuitable for commercial applications. The paper emphasizes the importance of Active Learning (AL) using expert labelers over crowdsourcing, which often lacks detailed insights into dataset structures. AL is an iterative process combining human labelers and AI models to optimize the labeling budget by intelligently selecting samples for human review. This approach addresses the challenge of handling large, constantly growing datasets that exceed available computational resources and memory. The paper presents a comprehensive data-centric framework for Machine Listening projects, detailing the configuration of recording nodes, database structure, and labeling budget optimization in resource-constrained scenarios. Applied to an industrial port in Valencia, Spain, the framework successfully labeled 6540 ten-second audio samples over five months with a small team, demonstrating its effectiveness and adaptability to various resource availability situations. Acknowledgments: The participation of Javier Naranjo-Alcazar, Jordi Grau-Haro and Pedro Zuccarello in this research was funded by the Valencian Institute for Business Competitiveness (IVACE) and the FEDER funds by means of project Soroll-IA2 (IMDEEA/2023/91). The research carried out for this publication has been partially funded by the project STARRING-NEURO (PID2022-137048OA-C44) funded by the Ministry of Science, Innovation and Universities of Spain and the European Union.
Paper Structure (9 sections, 6 figures)

This paper contains 9 sections, 6 figures.

Table of Contents

  1. Introduction
  2. Materials and Methods
  3. Recording node
  4. Active Learning
  5. Active Learning Database
  6. Audio Database
  7. Detailed Active Learning Framework
  8. Result
  9. Conclusion

Figures (6)

  • Figure 1: (a) Steps to be taken to solve Machine Listening problems with a Data-Centric perspective. (b) Recording node block diagram. (c) Recording node real ensemble
  • Figure 2: Audio Database structure
  • Figure 3: Framework for creating an audio database with a large amount of available data while optimizing the labeling budget
  • Figure 4: (a) 2D UMAP representation of an AL iteration. The orange dots represent the medoids used in the iteration, the green stars correspond to the audios proposed for labeling and the remaining dots are the audios discarded for labeling. (b) 2D UMAP representation top1 pre-trained PANNs prediction. The different colors show distinct classes. The circles mark areas of interest according to the AL process.
  • Figure 5: Histogram of tag frequencies for the 50 most repeated labels in the 6540 labeled audios
  • ...and 1 more figures