T2A-Feedback: Improving Basic Capabilities of Text-to-Audio Generation via Fine-grained AI Feedback

TL;DR

The paper addresses the difficulty of text-to-audio models to follow prompts and produce harmonious outputs in multi-event narratives. It introduces three fine-grained AI scoring pipelines—Event Occurrence, Event Sequence, and Acoustic&Harmonic Quality—to guide feedback learning, and builds the T2A-Feedback dataset and the T2A-EpicBench benchmark to evaluate advanced capabilities. Through preference tuning using these signals, the authors show improved alignment with human preferences on both simple and complex tasks, including emergent gains on long-caption scenarios. The work offers practical methods for scalable feedback-driven improvement in T2A and provides benchmarks that push the boundary of narrative audio generation.

Abstract

Text-to-audio (T2A) generation has achieved remarkable progress in generating a variety of audio outputs from language prompts. However, current state-of-the-art T2A models still struggle to satisfy human preferences for prompt-following and acoustic quality when generating complex multi-event audio. To improve the performance of the model in these high-level applications, we propose to enhance the basic capabilities of the model with AI feedback learning. First, we introduce fine-grained AI audio scoring pipelines to: 1) verify whether each event in the text prompt is present in the audio (Event Occurrence Score), 2) detect deviations in event sequences from the language description (Event Sequence Score), and 3) assess the overall acoustic and harmonic quality of the generated audio (Acoustic&Harmonic Quality). We evaluate these three automatic scoring pipelines and find that they correlate significantly better with human preferences than other evaluation metrics. This highlights their value as both feedback signals and evaluation metrics. Utilizing our robust scoring pipelines, we construct a large audio preference dataset, T2A-FeedBack, which contains 41k prompts and 249k audios, each accompanied by detailed scores. Moreover, we introduce T2A-EpicBench, a benchmark that focuses on long captions, multi-events, and story-telling scenarios, aiming to evaluate the advanced capabilities of T2A models. Finally, we demonstrate how T2A-FeedBack can enhance current state-of-the-art audio model. With simple preference tuning, the audio generation model exhibits significant improvements in both simple (AudioCaps test set) and complex (T2A-EpicBench) scenarios.

Figures (5)

• Figure 1: The overview of event occurrence and sequence scoring pipelines.
• Figure 2: Visualization of the predicted scores from our AI scoring pipeline. We highlight the first, second, and third events described in the captions using blue, brown, and green, respectively.
• Figure 3: Qualitative comparison between CLAP scores and EOS/ESS scores reveals distinct sensitivities to misalignment. By adding or reversing events in the ground-truth caption, the captions become misaligned with the audio in terms of event occurrence and sequence.
• Figure 4: Visualization of the impact of preference tuning with T2A-Feedback.
• Figure 5: Histograms of three different scores in T2A-Feedback.