Diffusion Models as Network Optimizers: Explorations and Analysis

TL;DR

The paper tackles IoT network optimization, which is challenging due to non-convex, mixed-integer, and multi-objective characteristics. It proposes diffusion-model-based optimizers (GDMs) that learn a distribution of high-quality solutions $p_{\theta}(\mathbf{y}_0|\mathbf{x})$ and sample from this distribution during inference to approach or reach the global optimum. A theoretical comparison demonstrates that generative models can provide tighter bounds on expected objective values than discriminative models, and the authors implement a DDPM with classifier-free guidance to enable conditional sampling. Through experiments on computation offloading, multi-channel sum-rate, and NOMA-UAV problems, the approach shows robust convergence to high-quality solutions and outperforms several baselines, highlighting its potential for complex IoT optimization tasks; code and data are provided for reproducibility. The work advances both theory and practice by clarifying why GDMs can serve as powerful optimizers and offering practical guidance on configuration, conditioning, and sampling in network settings.

Abstract

Network optimization is a fundamental challenge in the Internet of Things (IoT) network, often characterized by complex features that make it difficult to solve these problems. Recently, generative diffusion models (GDMs) have emerged as a promising new approach to network optimization, with the potential to directly address these optimization problems. However, the application of GDMs in this field is still in its early stages, and there is a noticeable lack of theoretical research and empirical findings. In this study, we first explore the intrinsic characteristics of generative models. Next, we provide a concise theoretical proof and intuitive demonstration of the advantages of generative models over discriminative models in network optimization. Based on this exploration, we implement GDMs as optimizers aimed at learning high-quality solution distributions for given inputs, sampling from these distributions during inference to approximate or achieve optimal solutions. Specifically, we utilize denoising diffusion probabilistic models (DDPMs) and employ a classifier-free guidance mechanism to manage conditional guidance based on input parameters. We conduct extensive experiments across three challenging network optimization problems. By investigating various model configurations and the principles of GDMs as optimizers, we demonstrate the ability to overcome prediction errors and validate the convergence of generated solutions to optimal solutions. We provide code and data at https://github.com/qiyu3816/DiffSG.

Figures (11)

• Figure 1: The learning results of generative models differ from those of discriminative models. Discriminative models focus on mapping $\mathbf{x}$ directly to the optimal solution, but they often incur learning errors that are difficult to eliminate. In contrast, generative models learn the distribution of high-quality solutions for a given input $\mathbf{x}$, with the optimal solution typically lying within this distribution.
• Figure 2: Forward noising process and reverse denoising process of GDM as an optimizer. The generation process involves randomness (represented by colored arrows) and ultimately converges to the target high-quality solution distribution.
• Figure 3: The effects of objective and constraint terms within the conditions.
• Figure 4: The effects of different condition strengths. Each bar includes upper and lower limit lines to indicate the range of variation observed across repeated experiments.
• Figure 5: The effects of the number of diffusion steps $T$.

Theorems & Definitions (1)

• Remark 1: better optimization bound of solution generation