Modeling Resilience of Collaborative AI Systems

TL;DR

Addresses resilience of CAIS during online learning under disruptive events, proposing a state-based framework and a FIFO windowed metric to track the Autonomous Classification Ratio ($ACR$). The method links $\epsilon$ and $K$ to autonomous decision-making and defines a sequence of resilience states (Steady, Disruptive, Recovered, Final) with concrete rules (State Length, PUT, PAT, HI Average) to quantify degradation and recovery. A real-world robotic demonstrator learning color classification validates the framework, showing repeatable cycles of disruption and recovery. The work provides practical metrics and automation to help CAIS managers monitor resilience and guide design of robust human–robot collaboration systems.

Abstract

A Collaborative Artificial Intelligence System (CAIS) performs actions in collaboration with the human to achieve a common goal. CAISs can use a trained AI model to control human-system interaction, or they can use human interaction to dynamically learn from humans in an online fashion. In online learning with human feedback, the AI model evolves by monitoring human interaction through the system sensors in the learning state, and actuates the autonomous components of the CAIS based on the learning in the operational state. Therefore, any disruptive event affecting these sensors may affect the AI model's ability to make accurate decisions and degrade the CAIS performance. Consequently, it is of paramount importance for CAIS managers to be able to automatically track the system performance to understand the resilience of the CAIS upon such disruptive events. In this paper, we provide a new framework to model CAIS performance when the system experiences a disruptive event. With our framework, we introduce a model of performance evolution of CAIS. The model is equipped with a set of measures that aim to support CAIS managers in the decision process to achieve the required resilience of the system. We tested our framework on a real-world case study of a robot collaborating online with the human, when the system is experiencing a disruptive event. The case study shows that our framework can be adopted in CAIS and integrated into the online execution of the CAIS activities.

Figures (5)

• Figure 1: Online Learning Process in CAIS.
• Figure 2: Framework execution states.
• Figure 3: CAIS Resilience Model.
• Figure 4: Collaborative Robot Learning from Demonstrations.
• Figure 5: Modeling of CAIS Performance Behavior.