Large-Scale Training Data Attribution for Music Generative Models via Unlearning

TL;DR

The paper addresses the challenge of attributing outputs of music generative models to their training data by presenting a white-box training data attribution method based on machine unlearning. It derives a practical unlearning objective anchored in the Fisher Information Matrix, yielding an update rule that approximates removing a generated sample from training, and applies this to a large-scale text-to-music diffusion model conditioned on CLAP embeddings. A diagonal FIM approximation enables scalable computation, and masking strategies are devised to handle padded silence in audio. Empirical evaluation on an in-house 115k-track dataset demonstrates feasible attribution with a grid-search tuning process and comparison to non-counterfactual methods, showing concentrated top-attribution signals and nuanced relationships to external embedding-based metrics. Overall, the work enables scalable, fair credit attribution for music AI and paves the way for more ethical and accountable AI-assisted music creation.

Abstract

This paper explores the use of unlearning methods for training data attribution (TDA) in music generative models trained on large-scale datasets. TDA aims to identify which specific training data points contributed the most to the generation of a particular output from a specific model. This is crucial in the context of AI-generated music, where proper recognition and credit for original artists are generally overlooked. By enabling white-box attribution, our work supports a fairer system for acknowledging artistic contributions and addresses pressing concerns related to AI ethics and copyright. We apply unlearning-based attribution to a text-to-music diffusion model trained on a large-scale dataset and investigate its feasibility and behavior in this setting. To validate the method, we perform a grid search over different hyperparameter configurations and quantitatively evaluate the consistency of the unlearning approach. We then compare attribution patterns from unlearning with non-counterfactual approaches. Our findings suggest that unlearning-based approaches can be effectively adapted to music generative models, introducing large-scale TDA to this domain and paving the way for more ethical and accountable AI systems for music creation.

Figures (2)

• Figure 1: Comparison of attribution scores from unlearning- and similarity-based methods. Mean (line) and standard deviation (shading) over attribution scores from 16 generated test samples. Min-max (left) and softmax (right) normalizations are shown (notice the logarithmic axes in the later).
• Figure 2: Correlation matrix between different attribution methods.