Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

KRAIL: A Knowledge-Driven Framework for Base Human Reliability Analysis Integrating IDHEAS and Large Language Models

Xingyu Xiao, Peng Chen, Ben Qi, Hongru Zhao, Jingang Liang, Jiejuan Tong, Haitao Wang

TL;DR

KRAIL tackles the bottleneck of base HEP estimation in human reliability analysis by fusing IDHEAS-DATA with large language models and retrieval-augmented generation. The authors propose a two-stage framework: a multi-agent task-decomposition stage and a knowledge-graph–driven integration stage, leveraging Neo4j and few-shot prompts to extract PIF, CFM, and related attributes before computing the base HEP. Empirical results on IDHEAS-derived data show that 5-shot configurations yield the best accuracy, with substantial time savings versus manual methods and robust ablation evidence for the value of the multi-agent design. The approach offers a scalable, semi-automated pathway to reliable HEP estimation under partial information, with potential for broader knowledge integration and domain expansion in safety-critical industries.

Abstract

Human reliability analysis (HRA) is crucial for evaluating and improving the safety of complex systems. Recent efforts have focused on estimating human error probability (HEP), but existing methods often rely heavily on expert knowledge,which can be subjective and time-consuming. Inspired by the success of large language models (LLMs) in natural language processing, this paper introduces a novel two-stage framework for knowledge-driven reliability analysis, integrating IDHEAS and LLMs (KRAIL). This innovative framework enables the semi-automated computation of base HEP values. Additionally, knowledge graphs are utilized as a form of retrieval-augmented generation (RAG) for enhancing the framework' s capability to retrieve and process relevant data efficiently. Experiments are systematically conducted and evaluated on authoritative datasets of human reliability. The experimental results of the proposed methodology demonstrate its superior performance on base HEP estimation under partial information for reliability assessment.

KRAIL: A Knowledge-Driven Framework for Base Human Reliability Analysis Integrating IDHEAS and Large Language Models

TL;DR

KRAIL tackles the bottleneck of base HEP estimation in human reliability analysis by fusing IDHEAS-DATA with large language models and retrieval-augmented generation. The authors propose a two-stage framework: a multi-agent task-decomposition stage and a knowledge-graph–driven integration stage, leveraging Neo4j and few-shot prompts to extract PIF, CFM, and related attributes before computing the base HEP. Empirical results on IDHEAS-derived data show that 5-shot configurations yield the best accuracy, with substantial time savings versus manual methods and robust ablation evidence for the value of the multi-agent design. The approach offers a scalable, semi-automated pathway to reliable HEP estimation under partial information, with potential for broader knowledge integration and domain expansion in safety-critical industries.

Abstract

Human reliability analysis (HRA) is crucial for evaluating and improving the safety of complex systems. Recent efforts have focused on estimating human error probability (HEP), but existing methods often rely heavily on expert knowledge,which can be subjective and time-consuming. Inspired by the success of large language models (LLMs) in natural language processing, this paper introduces a novel two-stage framework for knowledge-driven reliability analysis, integrating IDHEAS and LLMs (KRAIL). This innovative framework enables the semi-automated computation of base HEP values. Additionally, knowledge graphs are utilized as a form of retrieval-augmented generation (RAG) for enhancing the framework' s capability to retrieve and process relevant data efficiently. Experiments are systematically conducted and evaluated on authoritative datasets of human reliability. The experimental results of the proposed methodology demonstrate its superior performance on base HEP estimation under partial information for reliability assessment.

Paper Structure

This paper contains 22 sections, 11 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Literature Review
  3. Human Reliability Analysis
  4. Research Applications Utilizing IDHEAS-DATA
  5. Large Language Models and Retrieval-Augmented Generation
  6. Methodology
  7. Overview of the Proposed KRAIL
  8. Multi-Agent Framework for Task Decomposition
  9. Integration Framework for Base HEP Calculation
  10. Experimental Results
  11. Data Setup
  12. Experimental Setup
  13. Performance Across Different Shot Configurations
  14. Time Comparative Analysis: KRAIL vs. Manual
  15. Ablation Experiment
  16. ...and 7 more sections

Figures (11)

  • Figure 1: IDHEAS-ECA HRA Process xing2020integrated
  • Figure 2: KRAIL: A Knowledge-Driven Framework Integrating IDHEAS and Large Language Models for Base Human Reliability Analysis
  • Figure 3: Performance Across Different Shot Configurations for Varying Scenario Conditions.
  • Figure 4: Comparison of Time Spent Using Manual Methods vs. KRAIL Framework Across Different Scenarios
  • Figure 5: Venn Diagram of Ablation Study: Impact of Removing Multi-Agent on Accuracy. The intersection area represents the results that remained unchanged after removing the multi-agent component. The proportions labeled BETTER and WORSE reflect the accuracy changes in different elements after the removal of the multi-agent framework. Specifically, if, for a given case, the judgment for PIF changed from "Yes" to "No" after removing the multi-agent component, that case was classified as a bad case. The WORSE value in the figure represents the ratio of bad cases to the total number of distinct elements that showed a change in accuracy. The higher the BETTER value, the more effective our multi-agent framework is.
  • ...and 6 more figures