Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Enhancing Cluster Resilience: LLM-agent Based Autonomous Intelligent Cluster Diagnosis System and Evaluation Framework

Honghao Shi, Longkai Cheng, Wenli Wu, Yuhang Wang, Xuan Liu, Shaokai Nie, Weixv Wang, Xuebin Min, Chunlei Men, Yonghua Lin

TL;DR

An LLM-agent system designed to autonomously diagnose and resolve issues within AI clusters and demonstrated the superiority of the system in addressing the challenges faced in cluster diagnostics, particularly in detecting and rectifying performance issues more efficiently and accurately than traditional methods.

Abstract

Recent advancements in Large Language Models (LLMs) and related technologies such as Retrieval-Augmented Generation (RAG) and Diagram of Thought (DoT) have enabled the creation of autonomous intelligent systems capable of performing cluster diagnostics and troubleshooting. By integrating these technologies with self-play methodologies, we have developed an LLM-agent system designed to autonomously diagnose and resolve issues within AI clusters. Our innovations include a knowledge base tailored for cluster diagnostics, enhanced LLM algorithms, practical deployment strategies for agents, and a benchmark specifically designed for evaluating LLM capabilities in this domain. Through extensive experimentation across multiple dimensions, we have demonstrated the superiority of our system in addressing the challenges faced in cluster diagnostics, particularly in detecting and rectifying performance issues more efficiently and accurately than traditional methods.

Enhancing Cluster Resilience: LLM-agent Based Autonomous Intelligent Cluster Diagnosis System and Evaluation Framework

TL;DR

An LLM-agent system designed to autonomously diagnose and resolve issues within AI clusters and demonstrated the superiority of the system in addressing the challenges faced in cluster diagnostics, particularly in detecting and rectifying performance issues more efficiently and accurately than traditional methods.

Abstract

Recent advancements in Large Language Models (LLMs) and related technologies such as Retrieval-Augmented Generation (RAG) and Diagram of Thought (DoT) have enabled the creation of autonomous intelligent systems capable of performing cluster diagnostics and troubleshooting. By integrating these technologies with self-play methodologies, we have developed an LLM-agent system designed to autonomously diagnose and resolve issues within AI clusters. Our innovations include a knowledge base tailored for cluster diagnostics, enhanced LLM algorithms, practical deployment strategies for agents, and a benchmark specifically designed for evaluating LLM capabilities in this domain. Through extensive experimentation across multiple dimensions, we have demonstrated the superiority of our system in addressing the challenges faced in cluster diagnostics, particularly in detecting and rectifying performance issues more efficiently and accurately than traditional methods.

Paper Structure

This paper contains 30 sections, 3 figures, 4 tables.

Table of Contents

  1. INTRODUCTION
  2. RELATED WORKS
  3. LLM's Alignment and Enhancement
  4. AI-agent based Applications
  5. Diagnosis and Repair for AI Clusters
  6. SPECIAL TERMINOLOGIES
  7. Methods
  8. Overview
  9. Cluster Diagnosis Domain-specific Knowledge Base
  10. Diagnosis Dataset
  11. Performance Modeling
  12. Create LLM-agent with RAG-DoT-Selfplay techniques
  13. Using RAG to Build an LLM That Can Utilize External Knowledge
  14. Using DoT to Build an Agent That Can Reason and Plan
  15. Using Selfplay Techniques to Construct a Domain-specific MultiModal Agent
  16. ...and 15 more sections

Figures (3)

  • Figure 1: Overview of the Intelligent Maintenance System Based on LLM-Agents
  • Figure 2: Tools for LLM-agent to Diagnose AI Cluster
  • Figure 3: Multi-variable Task Performance Modeling. A shows compute-memory, B shows interconnect-memory, C shows interconnect-compute