Stable Consistency Tuning: Understanding and Improving Consistency Models

TL;DR

This work reframes consistency-model training through a Markov Decision Process and Temporal Difference learning lens, revealing fundamental trade-offs between consistency distillation and consistency training. Building on Easy Consistency Tuning, it introduces Stable Consistency Tuning (SCT) which employs a variance-reduced target via score identity, smoother training schedules, and multistep/inference enhancements, including classifier-free guidance and edge-skipping. Empirically, SCT delivers faster convergence and state-of-the-art 1-step and few-step FID scores on CIFAR-10 and ImageNet-64, highlighting its practical impact for fast, high-quality generation. The results offer a unified perspective for improving consistency models and point to scalable extensions toward more complex data regimes.

Abstract

Diffusion models achieve superior generation quality but suffer from slow generation speed due to the iterative nature of denoising. In contrast, consistency models, a new generative family, achieve competitive performance with significantly faster sampling. These models are trained either through consistency distillation, which leverages pretrained diffusion models, or consistency training/tuning directly from raw data. In this work, we propose a novel framework for understanding consistency models by modeling the denoising process of the diffusion model as a Markov Decision Process (MDP) and framing consistency model training as the value estimation through Temporal Difference~(TD) Learning. More importantly, this framework allows us to analyze the limitations of current consistency training/tuning strategies. Built upon Easy Consistency Tuning (ECT), we propose Stable Consistency Tuning (SCT), which incorporates variance-reduced learning using the score identity. SCT leads to significant performance improvements on benchmarks such as CIFAR-10 and ImageNet-64. On ImageNet-64, SCT achieves 1-step FID 2.42 and 2-step FID 1.55, a new SoTA for consistency models.

Figures (15)

• Figure 1: Stable consistency tuning (SCT) with variance reduced training target. SCT provides a unifying perspective to understand different training strategies of consistency models.
• Figure 2: Phasing the ODE path along the time axis for consistency training. We visualize both training and inference techniques in discrete form for easier understanding.
• Figure 3: FID vs Training iterations. SCT has faster convergence speed and better performance upper bound than ECT.
• Figure 4: The effectiveness of variance reduced training target.
• Figure 5: The effectiveness of edge-skipping multistep sampling.