Adaptive Fault Tolerance Mechanisms of Large Language Models in Cloud Computing Environments
Yihong Jin, Ze Yang, Xinhe Xu, Yihan Zhang, Shuyang Ji
TL;DR
This paper tackles the reliability and efficiency of Large Language Models deployed in cloud environments where hardware faults, network issues, and workload spikes hinder training and inference. It proposes an adaptive fault-tolerance framework that combines failure prediction using a multilayer perceptron to estimate $P(\text{fault}_t)$, dynamic checkpointing via $\lambda_t = \alpha P(\text{fault}_t) + \beta I_t$, and anomaly-mitigation guided by a Markov-state model and an optimization objective. Anomaly detection and recovery decisions are driven by a state-transition model $P(s_{t+1} | s_t) = \frac{e^{-\lambda |s_{t+1} - s_t|}}{Z_t}$ and an impact-cost function $L(s_t) = \lambda_1 \text{ResourceCost}(s_t) + \lambda_2 \text{FaultImpact}(s_t)$, enabling proactive failover to standby resources when thresholds are exceeded. Experiments on the DSTC dataset show the approach reduces downtime by about 30% and achieves higher fault-prediction accuracy than classical fault-tolerance methods, highlighting its potential for cloud-based LLM resilience.
Abstract
With the rapid evolution of Large Language Models (LLMs) and their large-scale experimentation in cloud-computing spaces, the challenge of guaranteeing their security and efficiency in a failure scenario has become a main issue. To ensure the reliability and availability of large-scale language models in cloud computing scenarios, such as frequent resource failures, network problems, and computational overheads, this study proposes a novel adaptive fault tolerance mechanism. It builds upon known fault-tolerant mechanisms, such as checkpointing, redundancy, and state transposition, introducing dynamic resource allocation and prediction of failure based on real-time performance metrics. The hybrid model integrates data driven deep learning-based anomaly detection technique underlining the contribution of cloud orchestration middleware for predictive prevention of system failures. Additionally, the model integrates adaptive checkpointing and recovery strategies that dynamically adapt according to load and system state to minimize the influence on the performance of the model and minimize downtime. The experimental results demonstrate that the designed model considerably enhances the fault tolerance in large-scale cloud surroundings, and decreases the system downtime by $\mathbf{30\%}$, and has a better modeling availability than the classical fault tolerance mechanism.