RESMETRIC: Analyzing Resilience to Enable Research on Antifragility

TL;DR

ResMetric addresses the lack of an out-of-the-box, model-agnostic tool to quantify resilience and antifragility in self-adaptive systems. It provides a Python package with a CLI to compute and visualize a broad set of resilience metrics from time-series performance data, including dip-agnostic metrics like AUC and thresholds, and dip-dependent metrics such as robustness, recovery rate, and an Integrated Resilience Metric (IRM); it also introduces antifragility measures via α_u and α_bar. The paper demonstrates ResMetric on a gas-delivery lifelong learning case study, comparing multiple learning models and showing how metric choices and dip-detection methods influence antifragility assessments. Key contributions include the modular, extensible design, the explicit treatment of post-dip improvements, and the practical demonstration that naive antifragility assessments can be misleading. The work provides a versatile, interpretable foundation for researchers and practitioners to select appropriate resilience concepts and to compare adaptation strategies across domains, contributing to more rigorous antifragility analyses in SASs.

Abstract

A key feature in self-adaptive systems is resilience, which is an ongoing research topic. Recently, the community started to explore antifragility, which describes the improvement of resilience over time. While there are model-agnostic resilience metrics, there is currently no out-of-the-box tool for researchers and practitioners to determine to which degree their system is resilient. To facilitate research on antifragility, we present ResMetric, a model-agnostic tool that calculates and visualizes various resilience metrics based on the quality of service over time. With ResMetric, researchers can evaluate their definition of resilience and antifragility. This paper highlights how ResMetric can be employed by demonstrating its use in a case study on gas detection.

Figures (8)

• Figure 1: Model of a system's performance during and after a disruption.
• Figure 2: ResMetric tool architecture.
• Figure 3: The performance quality of the four models investigated in the gas delivery case study.
• Figure 4: Resilience Metrics: Area Under Curve (AUC) (solid non-faded lines), and Time spent below threshold (dashed lines). The threshold is set at 0.8 (dashed black line).
• Figure 5: R, RR, and RL based on detected dips. Vertical dashed lines represent the maximum drop used for calculating, R, while the dotted lines and markers are for RR and RL. The plot on the right is a subselection of batches and methods to demonstrate the labeling. Robustness is at the maximum drops of dips, and the recovery rate/ability is labeled as a tuple.