Etude: Piano Cover Generation with a Three-Stage Approach -- Extract, strucTUralize, and DEcode

TL;DR

Etude tackles automatic piano cover generation by explicitly disentangling structure from note generation. It introduces a three-stage pipeline—Extract, Structuralize, and Decode—coupled with a Beat-Detector-derived rhythmic framework ($F_{beat}$) and a minimal Tiny-REMI token set to enforce structural consistency while enabling bar-level style control. Empirical results show state-of-the-art subjective quality and competitive objective metrics against strong baselines, with ablations confirming that architecture largely drives gains. The work provides a scalable, controllable framework for high-quality APCG and highlights avenues for improved beat tracking and evaluation methodologies.

Abstract

Piano cover generation aims to automatically transform a pop song into a piano arrangement. While numerous deep learning approaches have been proposed, existing models often fail to maintain structural consistency with the original song, likely due to the absence of beat-aware mechanisms or the difficulty of modeling complex rhythmic patterns. Rhythmic information is crucial, as it defines structural similarity (e.g., tempo, BPM) and directly impacts the overall quality of the generated music. In this paper, we introduce Etude, a three-stage architecture consisting of Extract, strucTUralize, and DEcode stages. By pre-extracting rhythmic information and applying a novel, simplified REMI-based tokenization, our model produces covers that preserve proper song structure, enhance fluency and musical dynamics, and support highly controllable generation through style injection. Subjective evaluations with human listeners show that Etude substantially outperforms prior models, achieving a quality level comparable to that of human composers.

Figures (3)

• Figure 1: The architecture of our proposed Etude framework, which comprises three main components: an Extractor for the extract stage, a Beat-Detector for the structuralize stage, and a Decoder for the decode stage.
• Figure 2: The overall architecture of Etude, where an Extractor derives harmonic features, a pre-trained Beat-Detector provides the rhythmic framework for tokenization and de-tokenization, and a Decoder receives these inputs along with style prompts. During training, the lower piano cover branch provides ground-truth target sequences ($Y$) aligned via weak-alignment. During inference, the lower branch content is replaced by autoregressively generated outputs: the Decoder generates each bar $Y_i$ from the corresponding feature bar $X_i$ and previous context. The Extractor and Decoder are trained separately.
• Figure 3: An example of Tiny-REMI tokenization, where the sequence is delimited by Bar [BOS] and Bar [EOS] tokens, and each note event group (Note + Dur) is preceded by a Pos token marking its relative position.