Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ChordSync: Conformer-Based Alignment of Chord Annotations to Music Audio

Andrea Poltronieri, Valentina Presutti, Martín Rocamora

TL;DR

ChordSync presents a novel conformer-based approach to align chord annotations with audio without requiring weak alignment, enabling the use of crowd-sourced chord data for MIR tasks. By combining a CQT-based audio representation, a Conformer acoustic model, and a CTC forced-alignment decoder, it achieves competitive alignment performance relative to DTW-based methods while avoiding the need for pre-aligned data. The work provides a pre-trained model and a library, demonstrates effectiveness across diverse genres, and highlights potential for expanding chord-annotated datasets and music education tools. Overall, ChordSync offers a scalable pathway to richer, temporally aligned harmonic annotations that can accelerate chord estimation and related MIR research.

Abstract

In the Western music tradition, chords are the main constituent components of harmony, a fundamental dimension of music. Despite its relevance for several Music Information Retrieval (MIR) tasks, chord-annotated audio datasets are limited and need more diversity. One way to improve those resources is to leverage the large number of chord annotations available online, but this requires aligning them with music audio. However, existing audio-to-score alignment techniques, which typically rely on Dynamic Time Warping (DTW), fail to address this challenge, as they require weakly aligned data for precise synchronisation. In this paper, we introduce ChordSync, a novel conformer-based model designed to seamlessly align chord annotations with audio, eliminating the need for weak alignment. We also provide a pre-trained model and a user-friendly library, enabling users to synchronise chord annotations with audio tracks effortlessly. In this way, ChordSync creates opportunities for harnessing crowd-sourced chord data for MIR, especially in audio chord estimation, thereby facilitating the generation of novel datasets. Additionally, our system extends its utility to music education, enhancing music learning experiences by providing accurately aligned annotations, thus enabling learners to engage in synchronised musical practices.

ChordSync: Conformer-Based Alignment of Chord Annotations to Music Audio

TL;DR

ChordSync presents a novel conformer-based approach to align chord annotations with audio without requiring weak alignment, enabling the use of crowd-sourced chord data for MIR tasks. By combining a CQT-based audio representation, a Conformer acoustic model, and a CTC forced-alignment decoder, it achieves competitive alignment performance relative to DTW-based methods while avoiding the need for pre-aligned data. The work provides a pre-trained model and a library, demonstrates effectiveness across diverse genres, and highlights potential for expanding chord-annotated datasets and music education tools. Overall, ChordSync offers a scalable pathway to richer, temporally aligned harmonic annotations that can accelerate chord estimation and related MIR research.

Abstract

In the Western music tradition, chords are the main constituent components of harmony, a fundamental dimension of music. Despite its relevance for several Music Information Retrieval (MIR) tasks, chord-annotated audio datasets are limited and need more diversity. One way to improve those resources is to leverage the large number of chord annotations available online, but this requires aligning them with music audio. However, existing audio-to-score alignment techniques, which typically rely on Dynamic Time Warping (DTW), fail to address this challenge, as they require weakly aligned data for precise synchronisation. In this paper, we introduce ChordSync, a novel conformer-based model designed to seamlessly align chord annotations with audio, eliminating the need for weak alignment. We also provide a pre-trained model and a user-friendly library, enabling users to synchronise chord annotations with audio tracks effortlessly. In this way, ChordSync creates opportunities for harnessing crowd-sourced chord data for MIR, especially in audio chord estimation, thereby facilitating the generation of novel datasets. Additionally, our system extends its utility to music education, enhancing music learning experiences by providing accurately aligned annotations, thus enabling learners to engage in synchronised musical practices.
Paper Structure (15 sections, 4 equations, 3 figures, 3 tables)

This paper contains 15 sections, 4 equations, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Our Contribution
  3. Related Work
  4. Audio-to-Score Alignment
  5. Lyrics-to-Audio Alignment
  6. Conformer-based Approaches
  7. Method
  8. Problem Statement
  9. Preprocessing
  10. Conformer-based Acoustic Model
  11. Forced Alignment
  12. Experiments
  13. Chord Changes Detection Evaluation
  14. Alignment Evaluation
  15. Discussion and Conclusion

Figures (3)

  • Figure 1: Basic schema of ChordSync: The model processes a list of chords alongside the audio signal, producing time-aligned chords as output.
  • Figure 2: Architecture of ChordSync: (i) The audio signal undergoes preprocessing to Constant-Q Transform (yellow box); (ii) The preprocessed audio serves as input for training the conformer-based acoustic model (blue box); and (iii) The model output probabilities, along with the list of chord labels for alignment, is fed into a CTC forced alignment module (green box), which outputs the aligned chord labels.
  • Figure 3: Workflow of the pre-processing applied to the chord labels. Chord labels are numerically encoded and upsampled to match the length of the CQT.