Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Audio Foundation Models Outperform Symbolic Representations for Piano Performance Evaluation

Jai Dhiman

TL;DR

The paper addresses objective piano performance evaluation across 19 perceptual dimensions and tests whether audio foundation models outperform symbolic MIDI representations by rendering identical MIDI with Pianoteq. It compares pretrained audio encoders MuQ and MERT against the symbolic PercePiano baseline, demonstrating that MuQ layers 9–12 with Pianoteq soundfont augmentation achieve $R^2 = 0.537$, a $55\%$ improvement over the symbolic baseline of $R^2 = 0.347$, with significance $p < 10^{-25}$. Cross-soundfont generalization ($R^2 = 0.534 \pm 0.075$), external difficulty correlation ($\rho = 0.623$), and multi-performer consistency support robustness beyond the training data. Analysis of audio-symbolic fusion shows high error correlation ($r = 0.738$), explaining why fusion yields limited gains and supporting an audio-only approach. The authors release the full training pipeline, pretrained models, and inference code to promote reproducibility and further research.

Abstract

Automated piano performance evaluation traditionally relies on symbolic (MIDI) representations, which capture note-level information but miss the acoustic nuances that characterize expressive playing. I propose using pre-trained audio foundation models, specifically MuQ and MERT, to predict 19 perceptual dimensions of piano performance quality. Using synthesized audio from PercePiano MIDI files (rendered via Pianoteq), I compare audio and symbolic approaches under controlled conditions where both derive from identical source data. The best model, MuQ layers 9-12 with Pianoteq soundfont augmentation, achieves R^2 = 0.537 (95% CI: [0.465, 0.575]), representing a 55% improvement over the symbolic baseline (R^2 = 0.347). Statistical analysis confirms significance (p < 10^-25) with audio outperforming symbolic on all 19 dimensions. I validate the approach through cross-soundfont generalization (R^2 = 0.534 +/- 0.075), difficulty correlation with an external dataset (rho = 0.623), and multi-performer consistency analysis. Analysis of audio-symbolic fusion reveals high error correlation (r = 0.738), explaining why fusion provides minimal benefit: audio representations alone are sufficient. I release the complete training pipeline, pretrained models, and inference code.

Audio Foundation Models Outperform Symbolic Representations for Piano Performance Evaluation

TL;DR

The paper addresses objective piano performance evaluation across 19 perceptual dimensions and tests whether audio foundation models outperform symbolic MIDI representations by rendering identical MIDI with Pianoteq. It compares pretrained audio encoders MuQ and MERT against the symbolic PercePiano baseline, demonstrating that MuQ layers 9–12 with Pianoteq soundfont augmentation achieve , a improvement over the symbolic baseline of , with significance . Cross-soundfont generalization (), external difficulty correlation (), and multi-performer consistency support robustness beyond the training data. Analysis of audio-symbolic fusion shows high error correlation (), explaining why fusion yields limited gains and supporting an audio-only approach. The authors release the full training pipeline, pretrained models, and inference code to promote reproducibility and further research.

Abstract

Automated piano performance evaluation traditionally relies on symbolic (MIDI) representations, which capture note-level information but miss the acoustic nuances that characterize expressive playing. I propose using pre-trained audio foundation models, specifically MuQ and MERT, to predict 19 perceptual dimensions of piano performance quality. Using synthesized audio from PercePiano MIDI files (rendered via Pianoteq), I compare audio and symbolic approaches under controlled conditions where both derive from identical source data. The best model, MuQ layers 9-12 with Pianoteq soundfont augmentation, achieves R^2 = 0.537 (95% CI: [0.465, 0.575]), representing a 55% improvement over the symbolic baseline (R^2 = 0.347). Statistical analysis confirms significance (p < 10^-25) with audio outperforming symbolic on all 19 dimensions. I validate the approach through cross-soundfont generalization (R^2 = 0.534 +/- 0.075), difficulty correlation with an external dataset (rho = 0.623), and multi-performer consistency analysis. Analysis of audio-symbolic fusion reveals high error correlation (r = 0.738), explaining why fusion provides minimal benefit: audio representations alone are sufficient. I release the complete training pipeline, pretrained models, and inference code.
Paper Structure (24 sections, 3 equations, 5 figures, 2 tables)

This paper contains 24 sections, 3 equations, 5 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Method
  4. Problem Formulation
  5. Architecture
  6. Training
  7. Data Augmentation via Soundfont Ensemble
  8. Experiments
  9. Dataset
  10. Baselines
  11. Main Results
  12. Ablation Studies
  13. Per-Dimension Analysis
  14. Validation
  15. Cross-Soundfont Generalization
  16. ...and 9 more sections

Figures (5)

  • Figure 1: Symbolic vs. audio paths for piano evaluation. Both derive from identical MIDI input, but audio representations leverage pretrained knowledge from millions of hours of music, providing inductive biases that improve prediction of perceptual qualities.
  • Figure 2: Model architecture. MIDI scores are rendered through multiple Pianoteq soundfonts for augmentation. The MuQ encoder extracts frame features from layers 9--12. Mean pooling aggregates temporal information, and a 2-layer MLP produces 19 dimension scores.
  • Figure 3: $R^2$ by dimension category comparing audio (MuQ) and symbolic (MIDI) approaches. Audio outperforms symbolic across all categories, with largest gains in Technical (timing) and Dynamics dimensions. Average per-dimension $R^2$: MuQ 0.601, Symbolic 0.347.
  • Figure 4: Per-dimension correlation with piece difficulty (PSyllabus, $n=508$). Overall Spearman $\rho = 0.623$ ($p < 10^{-50}$). Timing and valence show strongest correlations with difficulty.
  • Figure :