Meta-Reinforcement Learning with Discrete World Models for Adaptive Load Balancing

TL;DR

The paper tackles adaptive load balancing in operating systems under dynamic, heterogeneous workloads. It extends DreamerV3 with a recurrent policy and discrete world models to enable rapid meta-learning via RL², evaluated on Park OS environments against an A2C baseline. Results show the DreamerV3-based agent maintains high performance with minimal retraining and exhibits resilience to catastrophic forgetting across shifting workload distributions. This approach offers a scalable, memory-enabled RL framework for robust, real-time resource management in modern systems, with discrete latent representations driving improved stability and adaptability.

Abstract

We integrate a meta-reinforcement learning algorithm with the DreamerV3 architecture to improve load balancing in operating systems. This approach enables rapid adaptation to dynamic workloads with minimal retraining, outperforming the Advantage Actor-Critic (A2C) algorithm in standard and adaptive trials. It demonstrates robust resilience to catastrophic forgetting, maintaining high performance under varying workload distributions and sizes. These findings have important implications for optimizing resource management and performance in modern operating systems. By addressing the challenges posed by dynamic and heterogeneous workloads, our approach advances the adaptability and efficiency of reinforcement learning in real-world system management tasks.

Figures (1)

• Figure 1: Performance of A2C (Left) and DreamerV3 (Right) on Adaptive Load Balancing Tasks