Generative Models in Decision Making: A Survey

TL;DR

A comprehensive review of the application of generative models in decision-making tasks and proposes three key directions for advancing next-generation generative directive models: high-performance algorithms, large-scale generalized decision-making models, and self-evolving and adaptive models.

Abstract

In recent years, the exceptional performance of generative models in generative tasks has sparked significant interest in their integration into decision-making processes. Due to their ability to handle complex data distributions and their strong model capacity, generative models can be effectively incorporated into decision-making systems by generating trajectories that guide agents toward high-reward state-action regions or intermediate sub-goals. This paper presents a comprehensive review of the application of generative models in decision-making tasks. We classify seven fundamental types of generative models: energy-based models, generative adversarial networks, variational autoencoders, normalizing flows, diffusion models, generative flow networks, and autoregressive models. Regarding their applications, we categorize their functions into three main roles: controllers, modelers and optimizers, and discuss how each role contributes to decision-making. Furthermore, we examine the deployment of these models across five critical real-world decision-making scenarios. Finally, we summarize the strengths and limitations of current approaches and propose three key directions for advancing next-generation generative directive models: high-performance algorithms, large-scale generalized decision-making models, and self-evolving and adaptive models.

Figures (15)

• Figure 1: Research trends in generative models and their applications in decision-making (2000-2024). The bars represent the rough average annual number of papers, sourced from google scholar. The search includes titles associated with seven classic types of generative models and their applications in five real-world scenarios we details in Fig. \ref{['fig:applications']}.
• Figure 2: An overview of the survey. Specific sections are distinguished by different colors. Best viewed in color.
• Figure 3: Comparison of seven generative models in decision-making: training stability, generation diversity, and computational efficiency. Larger bubbles represent higher computational efficiency, with different models indicated by distinct colors. Best viewed in color.
• Figure 4: A diagrammatic depiction of diffusion models and one denoising step is illustrated yang2023diffusion.
• Figure 5: Illustration of the structure of a Generative Flow Network, as a pointed DAG over states $s$, with particles flowing along edges to represent the flow function bengio2021gflownet.