SongEcho: Towards Cover Song Generation via Instance-Adaptive Element-wise Linear Modulation

TL;DR

SongEcho is presented, which leverages Instance-Adaptive Element-wise Linear Modulation (IA-EiLM), a framework that incorporates controllable generation by improving both conditioning injection mechanism and conditional representation, and constructs Suno70k, a high-quality AI song dataset enriched with comprehensive annotations.

Abstract

Cover songs constitute a vital aspect of musical culture, preserving the core melody of an original composition while reinterpreting it to infuse novel emotional depth and thematic emphasis. Although prior research has explored the reinterpretation of instrumental music through melody-conditioned text-to-music models, the task of cover song generation remains largely unaddressed. In this work, we reformulate our cover song generation as a conditional generation, which simultaneously generates new vocals and accompaniment conditioned on the original vocal melody and text prompts. To this end, we present SongEcho, which leverages Instance-Adaptive Element-wise Linear Modulation (IA-EiLM), a framework that incorporates controllable generation by improving both conditioning injection mechanism and conditional representation. To enhance the conditioning injection mechanism, we extend Feature-wise Linear Modulation (FiLM) to an Element-wise Linear Modulation (EiLM), to facilitate precise temporal alignment in melody control. For conditional representations, we propose Instance-Adaptive Condition Refinement (IACR), which refines conditioning features by interacting with the hidden states of the generative model, yielding instance-adaptive conditioning. Additionally, to address the scarcity of large-scale, open-source full-song datasets, we construct Suno70k, a high-quality AI song dataset enriched with comprehensive annotations. Experimental results across multiple datasets demonstrate that our approach generates superior cover songs compared to existing methods, while requiring fewer than 30% of the trainable parameters. The code, dataset, and demos are available at https://github.com/lsfhuihuiff/SongEcho_ICLR2026.

Figures (5)

• Figure 1: Differences between other condition injection mechanisms and our approach. EiLM eliminates the need for separate learning of temporal alignment in (a) while offering more flexible modulation than (b). "$\oplus$" represents element-wise addition, and "$\odot$" represents element-wise multiplication.
• Figure 2: We employ a Diffusion Transformer (DiT) as the song generation backbone with a novel conditioning method, "IA-EiLM", for vocal melody control. A Pitch Extractor and Melody Encoder extract melody features, denoted as "$m$". The IA-EiLM module, integrated into each Transformer block, comprises two components: IACR and EiLM. "IACR" facilitates interaction between "$m$" and hidden states "$h_i$", refining melody condition, while "EiLM" modulates "$h_i$" into "$h_i^m$" with modulation parameters "$\gamma_i$" and "$\beta_i$", derived from the refined melody condition.
• Figure 3: We visualize the F0 contour extracted from the song, along with the word-level and phoneme-level timestamps produced by the Montreal Forced Aligner (MFA) mcauliffe2017montreal. The full lyrics used in the example are: “I had a horse named Hair, we weren't going nowhere.”
• Figure 4: Lyrics transcribed by Whisper radford2023robust with All-in-One yao2025songeval for SongEval yao2025songeval exhibit punctuation errors, including run-on sentences (orange), multiple clauses per line (blue), and incorrect sentence breaks (green), whereas Suno70k's native lyrics ensure each phrase is on a separate line.
• Figure 5: Attention map visualization of MuseControlLite tsaimusecontrollite under full-audio conditioning. The clear diagonal pattern indicates that the post-softmax attention matrix approximates an identity matrix.