Guided by the Plan: Enhancing Faithful Autoregressive Text-to-Audio Generation with Guided Decoding

TL;DR

The paper investigates autoregressive text-to-audio generation and discovers that early prefix tokens encode global semantic attributes, indicating implicit planning. It introduces Plan-Critic, a lightweight transformer-based critic trained with a Generalized Advantage Estimation–inspired objective to predict final instruction-following quality from partial sequences, enabling prefix-guided sampling. Through a prefix-first decoding strategy, Plan-Critic prunes low-potential trajectories and reallocates compute to promising seeds, achieving up to a 10-point improvement in CLAP over the AR baseline while maintaining a fixed token budget. The results demonstrate that strictly causal models can achieve strong semantic alignment with complex prompts when guided by explicit planning signals, bridging causal generation with global semantic control. Limitations include reliance on CLAP as a proxy metric, a fixed prefix-length assumption, and potential training-data biases, suggesting avenues for future work in more nuanced rewards and hierarchical planning.

Abstract

Autoregressive (AR) models excel at generating temporally coherent audio by producing tokens sequentially, yet they often falter in faithfully following complex textual prompts, especially those describing complex sound events. We uncover a surprising capability in AR audio generators: their early prefix tokens implicitly encode global semantic attributes of the final output, such as event count and sound-object category, revealing a form of implicit planning. Building on this insight, we propose Plan-Critic, a lightweight auxiliary model trained with a Generalized Advantage Estimation (GAE)-inspired objective to predict final instruction-following quality from partial generations. At inference time, Plan-Critic enables guided exploration: it evaluates candidate prefixes early, prunes low-fidelity trajectories, and reallocates computation to high-potential planning seeds. Our Plan-Critic-guided sampling achieves up to a 10-point improvement in CLAP score over the AR baseline-establishing a new state of the art in AR text-to-audio generation-while maintaining computational parity with standard best-of-N decoding. This work bridges the gap between causal generation and global semantic alignment, demonstrating that even strictly autoregressive models can plan ahead.

Figures (6)

• Figure 1: Instruction-following performance ($y$-axis) of different methods, measured by CLAP scores. The $x$-axis labels, $\text{Event}@N$, indicate prompts with $N$ distinct audio events, where a higher number of events corresponds to increased complexity.
• Figure 2: Upper row: Event counting regression results, with Kendall (K), Spearman (S), and Pearson (P) correlation coefficients reported between predicted and ground-truth counts. Lower row: t-SNE visualization of prefix representations, colored by the object category associated with each prefix. The confusion matrix for the classification results is also included. Takeaways: the non-trivial generalization performance of event counting and sounding object classification demonstrate the high correlation between the prefix tokens with posterior global attributes. Best viewed in color.
• Figure 3: Overall pipeline of the proposed plan-critic guided framework. ① illustrates the process of collecting training data for the plan-critic model. ② shows the training of the plan-critic model, where we conduct sparse GAE supervision with certain step $t$ is involved in loss (elaborate in Section \ref{['subsec:setting']}). ③ depicts how the trained critic guides the generator’s inference sampling.
• Figure 4: Upper row: Using prefix tokens from real audio to regress the total event count of the full audio. Low row: Event count regression using prefix tokens from decoded (reconstructed) audio.
• Figure 5: Given a set of prefix tokens, each is expanded into 1,000 postfix sequences. The two plots show the distributions of the mean and standard deviation of CLAP scores computed across the 1,000 generations per prefix, aggregated over 10,000 distinct prefixes. To simplify the figure, we randomly select and visualize 25 points for each event count.