Generative AI for Lyapunov Optimization Theory in UAV-based Low-Altitude Economy Networking

TL;DR

This paper addresses the challenge of achieving stable, high-performance control in UAV-based LAE networks under dynamic channel and resource conditions by leveraging Lyapunov optimization. It proposes a novel framework that couples generative diffusion models with reinforcement learning to solve Lyapunov drift-plus-penalty objectives, reducing instability risks and improving real-time decision quality. Through a UAV-based LAE case study, the approach demonstrates enhanced uplink rates and lower energy consumption compared to traditional methods, validating the practical viability of GenAI-enabled Lyapunov optimization. The work outlines future directions in problem transformation, real-time adaptation, and automated method integration, highlighting a pathway for scalable, stable, and data-efficient optimization in dynamic aerial networks.

Abstract

Lyapunov optimization theory has recently emerged as a powerful mathematical framework for solving complex stochastic optimization problems by transforming long-term objectives into a sequence of real-time short-term decisions while ensuring system stability. This theory is particularly valuable in unmanned aerial vehicle (UAV)-based low-altitude economy (LAE) networking scenarios, where it could effectively address inherent challenges of dynamic network conditions, multiple optimization objectives, and stability requirements. Recently, generative artificial intelligence (GenAI) has garnered significant attention for its unprecedented capability to generate diverse digital content. Extending beyond content generation, in this paper, we propose a framework integrating generative diffusion models with reinforcement learning to address Lyapunov optimization problems in UAV-based LAE networking. We begin by introducing the fundamentals of Lyapunov optimization theory and analyzing the limitations of both conventional methods and traditional AI-enabled approaches. We then examine various GenAI models and comprehensively analyze their potential contributions to Lyapunov optimization. Subsequently, we develop a Lyapunov-guided generative diffusion model-based reinforcement learning framework and validate its effectiveness through a UAV-based LAE networking case study. Finally, we outline several directions for future research.

Figures (5)

• Figure 1: An overview of conventional methods, encompassing convex optimization and heuristic algorithms, and traditional AI approaches, which include supervised learning and reinforcement learning, for addressing Lyapunov optimization problems.
• Figure 2: A summary of the foundational architectures of key GenAI models--Transformers, generative adversarial networks, variational autoencoders, and generative diffusion models--and their potential applications in solving Lyapunov optimization problems, focusing on principles and advantages.
• Figure 3: The proposed GDM-based reinforcement learning framework: In step 1, GDM generates the action $a(t)$ through the reverse process based on the current system state $s(t)$, the positional encoding of the denoising step $k$, and Gaussian noise $x(K)$. In step 2, the environment provides feedback in the form of a reward $r(t)$ and transitions to the next state $s(t+1)$. In step 3, the transition tuple $\langle s(t),a(t),r(t),s(t+1) \rangle$ is stored in the replay buffer for future sampling. In step 4, transitions are randomly sampled from the replay buffer to improve the policies of both GDM-based actor networks and MLP-based critic networks. In steps 5-7, the critic networks and actor networks are trained by minimizing the temporal difference error and maximizing the expected cumulative rewards, respectively. In step 8, the target actor network and critic network are partially updated to stabilize the training process.
• Figure 4: The training curve of the proposed GDM-based DDPG and the conventional DDPG method.
• Figure 5: Performance evaluation for the proposed framework. (a) User Average Transmission Rate versus Wireless Bandwidth. (b) UAV Propulsion Energy versus Time.