Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Building AI Agents for Autonomous Clouds: Challenges and Design Principles

Manish Shetty, Yinfang Chen, Gagan Somashekar, Minghua Ma, Yogesh Simmhan, Xuchao Zhang, Jonathan Mace, Dax Vandevoorde, Pedro Las-Casas, Shachee Mishra Gupta, Suman Nath, Chetan Bansal, Saravan Rajmohan

TL;DR

This paper addresses the lack of standardized frameworks for building and evaluating AIOps agents aimed at autonomous clouds. It proposes AIOpsLab, a principled framework with modular components, live-services execution, real fault injection, and an Agent–Cloud–Interface to orchestrate interactions between agents and cloud services. A case study demonstrates an LLM-based ReAct agent performing fault detection and mitigation on a DeathStarBench SocialNetwork deployment, achieving a Time-to-Detect of 14s and Time-to-Mitigate of 36s with 10–12 interactions at about $0.25. The work establishes design requirements and decisions to enable reproducible benchmarks and scalable evaluation, laying the groundwork for robust, reusable AIOps research and practical deployment of autonomous clouds.

Abstract

The rapid growth in the use of Large Language Models (LLMs) and AI Agents as part of software development and deployment is revolutionizing the information technology landscape. While code generation receives significant attention, a higher-impact application lies in using AI agents for operational resilience of cloud services, which currently require significant human effort and domain knowledge. There is a growing interest in AI for IT Operations (AIOps) which aims to automate complex operational tasks, like fault localization and root cause analysis, thereby reducing human intervention and customer impact. However, achieving the vision of autonomous and self-healing clouds through AIOps is hampered by the lack of standardized frameworks for building, evaluating, and improving AIOps agents. This vision paper lays the groundwork for such a framework by first framing the requirements and then discussing design decisions that satisfy them. We also propose AIOpsLab, a prototype implementation leveraging agent-cloud-interface that orchestrates an application, injects real-time faults using chaos engineering, and interfaces with an agent to localize and resolve the faults. We report promising results and lay the groundwork to build a modular and robust framework for building, evaluating, and improving agents for autonomous clouds.

Building AI Agents for Autonomous Clouds: Challenges and Design Principles

TL;DR

This paper addresses the lack of standardized frameworks for building and evaluating AIOps agents aimed at autonomous clouds. It proposes AIOpsLab, a principled framework with modular components, live-services execution, real fault injection, and an Agent–Cloud–Interface to orchestrate interactions between agents and cloud services. A case study demonstrates an LLM-based ReAct agent performing fault detection and mitigation on a DeathStarBench SocialNetwork deployment, achieving a Time-to-Detect of 14s and Time-to-Mitigate of 36s with 10–12 interactions at about $0.25. The work establishes design requirements and decisions to enable reproducible benchmarks and scalable evaluation, laying the groundwork for robust, reusable AIOps research and practical deployment of autonomous clouds.

Abstract

The rapid growth in the use of Large Language Models (LLMs) and AI Agents as part of software development and deployment is revolutionizing the information technology landscape. While code generation receives significant attention, a higher-impact application lies in using AI agents for operational resilience of cloud services, which currently require significant human effort and domain knowledge. There is a growing interest in AI for IT Operations (AIOps) which aims to automate complex operational tasks, like fault localization and root cause analysis, thereby reducing human intervention and customer impact. However, achieving the vision of autonomous and self-healing clouds through AIOps is hampered by the lack of standardized frameworks for building, evaluating, and improving AIOps agents. This vision paper lays the groundwork for such a framework by first framing the requirements and then discussing design decisions that satisfy them. We also propose AIOpsLab, a prototype implementation leveraging agent-cloud-interface that orchestrates an application, injects real-time faults using chaos engineering, and interfaces with an agent to localize and resolve the faults. We report promising results and lay the groundwork to build a modular and robust framework for building, evaluating, and improving agents for autonomous clouds.
Paper Structure (19 sections, 3 figures)

This paper contains 19 sections, 3 figures.

Table of Contents

  1. Introduction
  2. Background and Related Work
  3. AIOpsLab: A Principled Vision
  4. Requirements and Principles
  5. Design Decisions
  6. Bring your Services Alive
  7. Real Faults, Real Challenges
  8. Evaluate Across the Board
  9. Orchestrate the Agent-Cloud-Interface
  10. Abstract Environments, not agents
  11. Observe Everything, Everywhere, All at Once.
  12. System Architecture
  13. Orchestrator
  14. Service
  15. Workload Generator
  16. ...and 4 more sections

Figures (3)

  • Figure 1: System Architecture of AIOpsLab.The Orchestrator coordinates interactions between various system elements and serves as the Agent-Cloud-Interface (ACI). Agents engage with the Orchestrator to solve tasks, receiving a problem description, instructions, and relevant APIs. The Orchestrator generates diverse problems using the Workload and Fault Generators, injecting these into applications it can deploy. The deployed service has observability at multiple layers, providing telemetry, traces, and logs. The Orchestrator communicates with the service and the cloud using several tools such as Kubernetes, Helm, and even a Shell. Agents act via the Orchestrator, which executes them and updates the service's state. The Orchestrator evaluates the final solution using predefined metrics for the task.
  • Figure 2: Orchestrator shares a problem with APIs to be used as actions and an agent detects an anomaly in the logs.
  • Figure 3: Agent identifying the root-cause as a K8S misconfiguration fault. It then goes about mitigation by patching it.