Efficient Diffusion Models: A Survey

TL;DR

This survey systematically catalogs efficiency techniques for diffusion models, organizing methods into algorithm‑level, system‑level, and framework perspectives. It covers fundamental formulations (DDPM, score matching, SDE/ODE, flow matching), and surveys their implications across image, video, text, audio, and 3D generation. Key contributions include latent diffusion, training tricks, efficient fine‑tuning with LoRA/adapters/ControlNet, advanced sampling and distillation strategies, and system‑level optimizations such as hardware co‑design and caching. The work also discusses practical frameworks and outlines future directions like hybrid autoregressive models, improved guidance without CFG, and efficient attention for video diffusion, aiming to broaden the accessibility of diffusion models. A publicly maintained GitHub repository accompanies the survey to help researchers track and contribute to efficient diffusion research.

Abstract

Diffusion models have emerged as powerful generative models capable of producing high-quality contents such as images, videos, and audio, demonstrating their potential to revolutionize digital content creation. However, these capabilities come at the cost of their significant computational resources and lengthy generation time, underscoring the critical need to develop efficient techniques for practical deployment. In this survey, we provide a systematic and comprehensive review of research on efficient diffusion models. We organize the literature in a taxonomy consisting of three main categories, covering distinct yet interconnected efficient diffusion model topics from algorithm-level, system-level, and framework perspective, respectively. We have also created a GitHub repository where we organize the papers featured in this survey at https://github.com/AIoT-MLSys-Lab/Efficient-Diffusion-Model-Survey. We hope our survey can serve as a valuable resource to help researchers and practitioners gain a systematic understanding of efficient diffusion model research and inspire them to contribute to this important and exciting field.

Figures (30)

• Figure 1: Taxonomy of efficient diffusion model literature.
• Figure 2: Overview of forward SDE process and reverse SDE process song2020score.
• Figure 5: Summary of efficient training techniques for diffusion models.
• Figure 6: Illustration of the rectified flow.
• Figure 7: Illustration of data-dependent adaptive priors for diffusion processes across different modalities.