Quantitative Measurement of Cyber Resilience: Modeling and Experimentation

TL;DR

This work tackles the challenge of quantifying cyber resilience in cyber-physical systems by formalizing resilience through goal achievement, introducing malware and bonware as competing time-varying effects, and defining the resilience metric $R$ as the ratio of areas under time-varying performance curves. It combines a parsimonious deterministic modeling framework—including constant, piecewise constant, LTV, and PLTV variants—with an integrated physical-digital test bed that links real CAN-based ECUs (PASTA) to a high-fidelity simulation (Unity) via the Active Defense Framework (ADF) and automated data collection (OpenTAP). The key contributions are (i) explicit definitions and solutions for how malware and bonware shape functionality over time, (ii) an experimental methodology that yields quantitative, repeatable resilience data from controlled cyber-attacks, and (iii) practical resilience summaries, including the $R$ measure and model-based malware/bonware effectiveness parameters $\mathcal{M}$ and $\mathcal{B}$. This framework enables robust, scalable assessment of cyber resilience in trucks and can be extended to other cyber-physical systems, potentially informing design and defense strategies. The work advances both theory and practice by providing a concrete, data-driven approach to measure and compare cyber resilience across terrains, weights, and attack scenarios, with a clear path to broader adoption via the accompanying methodology.

Abstract

Cyber resilience is the ability of a system to resist and recover from a cyber attack, thereby restoring the system's functionality. Effective design and development of a cyber resilient system requires experimental methods and tools for quantitative measuring of cyber resilience. This paper describes an experimental method and test bed for obtaining resilience-relevant data as a system (in our case -- a truck) traverses its route, in repeatable, systematic experiments. We model a truck equipped with an autonomous cyber-defense system and which also includes inherent physical resilience features. When attacked by malware, this ensemble of cyber-physical features (i.e., "bonware") strives to resist and recover from the performance degradation caused by the malware's attack. We propose parsimonious mathematical models to aid in quantifying systems' resilience to cyber attacks. Using the models, we identify quantitative characteristics obtainable from experimental data, and show that these characteristics can serve as useful quantitative measures of cyber resilience.

Figures (9)

• Figure 1: The functionality $F(t)$ of a system is plotted in yellow and normal functionality is plotted in light blue. In this example, there is a cyber attack at time $t_0$ which degrades functionality, and the system's resilience allows the functionality to begin recovering at time $t_1$. The area under the curve (AUC) is defined as the normalized area under the curve from time $t_0$ to time $T$, the end time of measured system performance.
• Figure 2: Normalized functionality, ${{F}}(t)/F_\text{N}{}$, is shown for various constant values of $\mathcal{M}$ (malware attacking) and $\mathcal{B}$ (bonware defending) and with initial condition ${{F}}(0)=F_\text{N}{}$. The functionality over time depends on the relative strengths of bonware and malware. With the system initially at normal functionality and malware effectiveness nonzero, functionality exhibits exponential decay.
• Figure 3: Normalized functionality, ${{F}}(t)/F_\text{N}{}$, is shown for piecewise linear time varying models and initial condition $F(0)=F_\text{N}{}$. Both malware and bonware effectivenesses are initially linear functions of time: $\mathcal{M}~=~\max(0.5-0.1t,0), \mathcal{B} = b_0+0.04t.$ When malware effectiveness reaches $\mathcal{M}=0$, bonware effectiveness continues to increase.
• Figure 4: A high-level overview of the data flow between components. Portions are derived from toyama2018pasta. Unity handles the interactions between the simulated trucks and the terrain, provides driver inputs to maintain pathing and speed over a pre-determined course across the terrain, and simulates many aspects of the vehicle to provide high-fidelity vehicle performance. PASTA is hardware-in-the-loop that generates actual CAN bus traffic in response to the inputs from Unity, and provides three vehicle ECUs (powertrain, body, and chassis). ADF allows for communication translation between Unity and PASTA, provides simulated ECU plugins for additional sensor information not present in PASTA, and implements simulated cyber attacks and defenses.
• Figure 5: Examples of experimental data, illustrating that cyber attacks reduce performance both in fuel efficiency (top left panel) and speed (bottom left panel), and that changes in cargo weight reduce fuel efficiency in the expected manner (right panel). Top left panel: The fuel efficiency of a heavy truck, carrying no cargo, during a run on hilly terrain. The orange curve indicates the fuel efficiency in the "engine ECU attack" run, which is contrasted with the (partly occluded) cyan curve that indicates the baseline run. Bottom left panel: Recorded speed during the same run. Right panel: Fuel efficiency, now for all four cargo conditions (from top to bottom: 0, 3,000, 6,000, and 9,000kg).