Pairing Real-Time Piano Transcription with Symbol-level Tracking for Precise and Robust Score Following

TL;DR

The paper argues that real-time score following benefits from transforming performance into the symbolic domain and aligning it with a symbolic score. It introduces a two-component system—a real-time AMT module based on a lightweight online transcription model and a novel symbol-level online time warping tracker with a pairwise pitch-time distance metric and tempo-aware path tracking. Empirical results on a piano corpus show that the symbolic approach can surpass audio-only OLTW in robustness and precision, even when transcription is imperfect, and can provide a strong upper bound when transcription is near-perfect. The work demonstrates the feasibility and advantages of a transcribe-and-track pipeline for real-time accompaniment and score following, while highlighting that further gains depend on improvements in real-time AMT latency and accuracy. Overall, this symbol-level tracking framework offers a principled route to robust real-time alignment between performances and symbolic scores.

Abstract

Real-time music tracking systems follow a musical performance and at any time report the current position in a corresponding score. Most existing methods approach this problem exclusively in the audio domain, typically using online time warping (OLTW) techniques on incoming audio and an audio representation of the score. Audio OLTW techniques have seen incremental improvements both in features and model heuristics which reached a performance plateau in the past ten years. We argue that converting and representing the performance in the symbolic domain -- thereby transforming music tracking into a symbolic task -- can be a more effective approach, even when the domain transformation is imperfect. Our music tracking system combines two real-time components: one handling audio-to-note transcription and the other a novel symbol-level tracker between transcribed input and score. We compare the performance of this mixed audio-symbolic approach with its equivalent audio-only counterpart, and demonstrate that our method outperforms the latter in terms of both precision, i.e., absolute tracking error, and robustness, i.e., tracking success.