Enhancing Cloud Network Resilience via a Robust LLM-Empowered Multi-Agent Reinforcement Learning Framework

TL;DR

CyberOps-Bots addresses cloud-network resilience under dynamic topology, scale, and evolving attack strategies by integrating a hierarchical LLM-based planner with heterogeneous pre-trained RL agents for localized defense, all under HITL supervision. The framework formally models defense as a multi-layer MDP with semantic perception, ReAct-based planning, long- and short-term memory for attack chains, and auditable decision logs. Empirical results on realistic cloud datasets show substantial gains in availability and rapid adaptation to changing conditions, confirming the method’s effectiveness and interpretability for secure cloud management. This work advances autonomous, robust cloud defense by combining semantic high-level reasoning with reliable low-level action execution while enabling expert intervention and auditability in real time.

Abstract

While virtualization and resource pooling empower cloud networks with structural flexibility and elastic scalability, they inevitably expand the attack surface and challenge cyber resilience. Reinforcement Learning (RL)-based defense strategies have been developed to optimize resource deployment and isolation policies under adversarial conditions, aiming to enhance system resilience by maintaining and restoring network availability. However, existing approaches lack robustness as they require retraining to adapt to dynamic changes in network structure, node scale, attack strategies, and attack intensity. Furthermore, the lack of Human-in-the-Loop (HITL) support limits interpretability and flexibility. To address these limitations, we propose CyberOps-Bots, a hierarchical multi-agent reinforcement learning framework empowered by Large Language Models (LLMs). Inspired by MITRE ATT&CK's Tactics-Techniques model, CyberOps-Bots features a two-layer architecture: (1) An upper-level LLM agent with four modules--ReAct planning, IPDRR-based perception, long-short term memory, and action/tool integration--performs global awareness, human intent recognition, and tactical planning; (2) Lower-level RL agents, developed via heterogeneous separated pre-training, execute atomic defense actions within localized network regions. This synergy preserves LLM adaptability and interpretability while ensuring reliable RL execution. Experiments on real cloud datasets show that, compared to state-of-the-art algorithms, CyberOps-Bots maintains network availability 68.5% higher and achieves a 34.7% jumpstart performance gain when shifting the scenarios without retraining. To our knowledge, this is the first study to establish a robust LLM-RL framework with HITL support for cloud defense. We will release our framework to the community, facilitating the advancement of robust and autonomous defense in cloud networks.

Figures (10)

• Figure 1: While technologies like virtualization and elastic scaling provide dynamic flexibility, they inevitably expand the attack surface. This increased exposure facilitates specific cloud attacks, such as container escape and malware in cloud storage, thereby challenging system resilience.
• Figure 2: A typical cloud-native e-commerce architecture, exemplifying the four dynamic aspects (A1-A4). i) The network frequently performs elastic scaling and instance migration in response to workload fluctuations. ii) Business instances dynamically expand as new features are deployed or partners integrated. iii) Meanwhile, attack tactics and scale are constantly evolving, from initial port scanning of public entry points, to compromising databases via vulnerable middleware, and ultimately escalating into coordinated DDoS attacks that cripple core services.
• Figure 3: The CyberOps-Bots framework architecture, comprising three coordinated layers: (i) the Env Layer simulating a dynamic adversarial cloud network; (ii) the LLM Layer with Perception, Planning, Memory, and Action modules for semantic reasoning and HITL support; and (iii) the RL Layer of pre-trained heterogeneous agents executing localized defense actions, enabling adaptive response without retraining.
• Figure 4: Experimental setup for evaluating the adaptability of CyberOps-Bots and baseline algorithms.
• Figure 5: Figure (a-c) present the experimental results when the test scenario is dynamically switched from Sce1 (a) to Sce4 (b) and Sce7 (c), showing the variation of average cumulative rewards over decision steps. Figure (d) illustrates the jumpstart performance of each algorithm during scenario switching, highlighting their ability to adapt quickly to new environments.

Theorems & Definitions (11)

• Definition 1
• Definition 2
• Definition 3
• Definition 4
• Definition 5
• Definition 6
• Definition 7
• Definition 8
• Definition 9
• Definition 10