Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Reinforcement Learning-based Adaptive Mitigation of Uncorrected DRAM Errors in the Field

Isaac Boixaderas, Sergi Moré, Javier Bartolome, David Vicente, Petar Radojković, Paul M. Carpenter, Eduard Ayguadé

TL;DR

We address the cost-inefficiency of mitigating uncorrected DRAM errors in large HPC clusters by introducing a reinforcement learning-based policy that adaptively triggers mitigations. Framed as a Markov decision process and solved with a dueling double deep Q-network and prioritized experience replay, the agent weighs the likelihood and potential cost of uncorrected errors to decide when to mitigate. Evaluated on two years of MareNostrum production logs, the method achieves a 54% reduction in lost compute time versus never-mitigate, and comes within a small margin of an Oracle-based optimal policy while incurring substantially lower mitigation costs. The approach generalizes across DRAM manufacturers and job sizes, is validated via time-series nested cross-validation, and is released as open-source, enabling deployment on other HPC systems without extensive tuning.

Abstract

Scaling to larger systems, with current levels of reliability, requires cost-effective methods to mitigate hardware failures. One of the main causes of hardware failure is an uncorrected error in memory, which terminates the current job and wastes all computation since the last checkpoint. This paper presents the first adaptive method for triggering uncorrected error mitigation. It uses a prediction approach that considers the likelihood of an uncorrected error and its current potential cost. The method is based on reinforcement learning, and the only user-defined parameters are the mitigation cost and whether the job can be restarted from a mitigation point. We evaluate our method using classical machine learning metrics together with a cost-benefit analysis, which compares the cost of mitigation actions with the benefits from mitigating some of the errors. On two years of production logs from the MareNostrum supercomputer, our method reduces lost compute time by 54% compared with no mitigation and is just 6% below the optimal Oracle method. All source code is open source.

Reinforcement Learning-based Adaptive Mitigation of Uncorrected DRAM Errors in the Field

TL;DR

We address the cost-inefficiency of mitigating uncorrected DRAM errors in large HPC clusters by introducing a reinforcement learning-based policy that adaptively triggers mitigations. Framed as a Markov decision process and solved with a dueling double deep Q-network and prioritized experience replay, the agent weighs the likelihood and potential cost of uncorrected errors to decide when to mitigate. Evaluated on two years of MareNostrum production logs, the method achieves a 54% reduction in lost compute time versus never-mitigate, and comes within a small margin of an Oracle-based optimal policy while incurring substantially lower mitigation costs. The approach generalizes across DRAM manufacturers and job sizes, is validated via time-series nested cross-validation, and is released as open-source, enabling deployment on other HPC systems without extensive tuning.

Abstract

Scaling to larger systems, with current levels of reliability, requires cost-effective methods to mitigate hardware failures. One of the main causes of hardware failure is an uncorrected error in memory, which terminates the current job and wastes all computation since the last checkpoint. This paper presents the first adaptive method for triggering uncorrected error mitigation. It uses a prediction approach that considers the likelihood of an uncorrected error and its current potential cost. The method is based on reinforcement learning, and the only user-defined parameters are the mitigation cost and whether the job can be restarted from a mitigation point. We evaluate our method using classical machine learning metrics together with a cost-benefit analysis, which compares the cost of mitigation actions with the benefits from mitigating some of the errors. On two years of production logs from the MareNostrum supercomputer, our method reduces lost compute time by 54% compared with no mitigation and is just 6% below the optimal Oracle method. All source code is open source.
Paper Structure (42 sections, 6 equations, 7 figures, 2 tables)

This paper contains 42 sections, 6 equations, 7 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Environment description
  3. MareNostrum 3 error logs
  4. Corrected errors (CEs)
  5. Uncorrected errors (UEs)
  6. UE reduction
  7. DIMM retirement bias
  8. Quantitative analysis
  9. MareNostrum 4 job logs
  10. Adaptive Error Mitigation Control
  11. Background
  12. MDP formulation of UE mitigation control
  13. State and features
  14. Actions
  15. State transitions
  16. ...and 27 more sections

Figures (7)

  • Figure 1: Interaction between RL agent and environment for adaptive UE mitigation.
  • Figure 2: Evaluation using time series nested cross-validation. The error log is divided into six equal parts and evaluation for each split is divided into training (with multiple hyperparame- ters), validation (to find the best hyperparameters), and testing.
  • Figure 3: Total cost for MN/All, as the sum of UE cost (solid color) and mitigation cost (with dashes). The RL agent has lower total cost than the other approaches due to a much lower mitigation cost. Unlike SC20-RF it is not sensitive to a user-supplied parameter. The results are stable for mitigation costs between 2 node–minutes and 10 node–minutes.
  • Figure 4: Time series nested cross-validation for MN/All with 2 node--minute mitigation cost, starting from untrained models. The total cost is the sum of UE cost (solid color) and mitigation cost (with dashes).
  • Figure 5: Total cost for the three anonymized DRAM manufacturers. Each bar is the sum of UE cost (solid color) and mitigation cost (with dashes). The RL agent has lower total cost than the other approaches due to a much lower mitigation cost. This result is consistent across all DRAM manufacturers.
  • ...and 2 more figures