Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

DARC: Drum accompaniment generation with fine-grained rhythm control

Trey Brosnan

TL;DR

DARC introduces rhythm-conditioned drum accompaniment by conditioning a STAGE-based generator on both musical context and a rhythm prompt using NMF-derived timbre-class activations. The approach uses parameter-efficient fine-tuning with jump and adaptive in-attention and encodes rhythm with an NMF representation to preserve timbre classes while tracking onsets. Experiments reveal promising feasibility but are hindered by audio fidelity and evaluation metric limitations, underscoring the need for higher-quality data and robust rhythm/coherence metrics. The work positions DARC between timbre-transfer and stem-generation, offering a tool for rapid musical prototyping and co-creation with controllable rhythm input.

Abstract

In music creation, rapid prototyping is essential for exploring and refining ideas, yet existing generative tools often fall short when users require both structural control and stylistic flexibility. Prior approaches in stem-to-stem generation can condition on other musical stems but offer limited control over rhythm, and timbre-transfer methods allow users to specify specific rhythms, but cannot condition on musical context. We introduce DARC, a generative drum accompaniment model that conditions both on musical context from other stems and explicit rhythm prompts such as beatboxing or tapping tracks. Using parameter-efficient fine-tuning, we augment STAGE, a state-of-the-art drum stem generator, with fine-grained rhythm control while maintaining musical context awareness.

DARC: Drum accompaniment generation with fine-grained rhythm control

TL;DR

DARC introduces rhythm-conditioned drum accompaniment by conditioning a STAGE-based generator on both musical context and a rhythm prompt using NMF-derived timbre-class activations. The approach uses parameter-efficient fine-tuning with jump and adaptive in-attention and encodes rhythm with an NMF representation to preserve timbre classes while tracking onsets. Experiments reveal promising feasibility but are hindered by audio fidelity and evaluation metric limitations, underscoring the need for higher-quality data and robust rhythm/coherence metrics. The work positions DARC between timbre-transfer and stem-generation, offering a tool for rapid musical prototyping and co-creation with controllable rhythm input.

Abstract

In music creation, rapid prototyping is essential for exploring and refining ideas, yet existing generative tools often fall short when users require both structural control and stylistic flexibility. Prior approaches in stem-to-stem generation can condition on other musical stems but offer limited control over rhythm, and timbre-transfer methods allow users to specify specific rhythms, but cannot condition on musical context. We introduce DARC, a generative drum accompaniment model that conditions both on musical context from other stems and explicit rhythm prompts such as beatboxing or tapping tracks. Using parameter-efficient fine-tuning, we augment STAGE, a state-of-the-art drum stem generator, with fine-grained rhythm control while maintaining musical context awareness.
Paper Structure (17 sections, 1 figure, 2 tables)

This paper contains 17 sections, 1 figure, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Accompaniment Generation
  4. Tap2Drum Generation
  5. Method
  6. Overview
  7. Rhythm Feature Representation
  8. Fine-Tuning
  9. Experimental Setup
  10. Audio Quality
  11. Rhythm Prompt Adherence
  12. Musical Coherence
  13. Results and Discussion
  14. Rhythm Prompt Adherence
  15. Musical Coherence
  16. ...and 2 more sections

Figures (1)

  • Figure 1: Architecture of the proposed rhythm-conditioned music generation model. Musical context and rhythm prompt are provided as audio inputs. The tokenized musical context is prepended to the input sequence, and the rhythm prompt is transcribed into (onset time, timbre class) pairs using non-negative matrix factorization (NMF). The rhythm embedding is passed through the self-attention layers via jump fine-tuning and adaptive in-attention musiccongen. The model outputs EnCodec audio tokens that are decoded to the final waveform.