Diffusion Models are Evolutionary Algorithms

TL;DR

The Diffusion Evolution method is proposed, an evolutionary algorithm utilizing iterative denoising -- as originally introduced in the context of diffusion models -- to heuristically refine solutions in parameter spaces and efficiently identifies multiple optimal solutions and outperforms prominent mainstream evolutionary algorithms.

Abstract

In a convergence of machine learning and biology, we reveal that diffusion models are evolutionary algorithms. By considering evolution as a denoising process and reversed evolution as diffusion, we mathematically demonstrate that diffusion models inherently perform evolutionary algorithms, naturally encompassing selection, mutation, and reproductive isolation. Building on this equivalence, we propose the Diffusion Evolution method: an evolutionary algorithm utilizing iterative denoising -- as originally introduced in the context of diffusion models -- to heuristically refine solutions in parameter spaces. Unlike traditional approaches, Diffusion Evolution efficiently identifies multiple optimal solutions and outperforms prominent mainstream evolutionary algorithms. Furthermore, leveraging advanced concepts from diffusion models, namely latent space diffusion and accelerated sampling, we introduce Latent Space Diffusion Evolution, which finds solutions for evolutionary tasks in high-dimensional complex parameter space while significantly reducing computational steps. This parallel between diffusion and evolution not only bridges two different fields but also opens new avenues for mutual enhancement, raising questions about open-ended evolution and potentially utilizing non-Gaussian or discrete diffusion models in the context of Diffusion Evolution.

Figures (4)

• Figure 1: Evolution processes can be viewed as the inverse process of diffusion, where higher fitness populations (red points) have higher final probability density. The initially unstructured parameters are iteratively refined towards high-fitness regions in parameter space.
• Figure 2: (a) Diffusion Evolution on a two-peak fitness landscape: Populations near the two black crosses have higher fitness. For each individual ${\bm{x}}_t$ (black star), its target ${\hat{{\bm{x}}}}_0$ (red dots) is estimated by a weighted average of its neighbors (c.f., dots within the blue disks, respectively); larger dot-size indicates higher fitness. The individual then moves a small step forward to the next generation (orange star). As evolution proceeds, the neighbor range decreases, making the process increasingly sensitive to local neighbors, thereby enabling global competition originally, while "zooming in" eventually to balance between optimization and diversity. (b) By mapping the population to a 1-D space (dashed lines in (a)), we track the progress of Diffusion Evolution. As evolution progresses, both the individuals (gray) and their estimated origins (red) move closer to the targets (vertical dashed lines), with the estimated origins advancing faster.
• Figure 3: One of the benchmark experiments (columns) on different fitness functions with selected evolutionary algorithms (rows). The blue regions represent high fitness, while white indicates low fitness in a two-dimensional parameter space. All fitness functions are mapped to a 0 to 1 range to make them comparable, as detailed in the Appendix. The Diffusion Evolution algorithm finds multiple optimal solutions in the 2D benchmarks while maintaining genetic diversity. Red dots indicating the final population and gray lines show the trajectories of populations; for simplicity, only the trajectories of 64 individuals are plotted here. In the CMAES and PEPG experiments, the gray ellipsoids represent the estimated covariances at each step, and the gray lines represent the history of estimated averages. The red dots indicate the final population. In the OpenES experiments, the red dots indicate the final population, and the gray line represents the parameter trajectory.
• Figure 4: (a) Evolution process of cart-pole tasks: The horizontal axis shows survival time, and the vertical axis represents generations. Each point indicates an individual's state (pole angle, cart shift) at their final survival. As the evolution progresses, more systems survive longer and achieve higher rewards. (b) Compared to the original Diffusion Evolution (blue), the Latent Space Diffusion Evolution method (red) significantly improves performance, while the CMA-ES method (gray) fails to find any solutions in given generations. This latent method can even be applied to high-dimensional spaces (orange), with dimensions as high as 17,410. Each experiment is repeated 100 times, with medians (solid lines) and ranges (25% to 75% quantile) shown as shaded areas. (c) Projecting the parameters of individuals into a latent space visualize their diversity. The same projection is used for all results (except for the high-dimensional experiment, which has a different original dimension). This indicates enhanced diversity with the latent method. (d) The cart-pole system consists of a pole hinged to the cart. And the controller balances the pole by moving the cart left or right.