Towards the Best Solution for Complex System Reliability: Can Statistics Outperform Machine Learning?

TL;DR

It is shown that in many practical applications, traditional statistical algorithms frequently produce more accurate and interpretable results compared with black-box machine learning methods.

Abstract

Studying the reliability of complex systems using machine learning techniques involves facing a series of technical and practical challenges, ranging from the intrinsic nature of the system and data to the difficulties in modeling and effectively deploying models in real-world scenarios. This study compares the effectiveness of classical statistical techniques and machine learning methods for improving complex system analysis in reliability assessments. We aim to demonstrate that classical statistical algorithms often yield more precise and interpretable results than black-box machine learning approaches in many practical applications. The evaluation is conducted using both real-world data and simulated scenarios. We report the results obtained from statistical modeling algorithms, as well as from machine learning methods including neural networks, K-nearest neighbors, and random forests.

Figures (7)

• Figure 1: Splitting datasets.
• Figure 2: Confusion matrix.
• Figure 3: FA-LR-IS algorithm.
• Figure 4: Reliability Block Diagrams for the simulated systems.
• Figure 5: Boxplot comparing $AUC$ between models.