Perspectives-Observer-Transparency -- A Novel Paradigm for Modelling the Human in Human-To-Anything Interaction Based on a Structured Review of the Human Digital Twin

TL;DR

The paper tackles the gap between monitoring and understanding human behavior in H2X systems, arguing that existing HDTs are largely mechanistic and insufficient for modeling inner states. It introduces Perspectives-Observer-Transparency, a two-perspective framework with bridge models and observer-based transparency mechanism, validated conceptually on two HSIs scenarios. The authors perform a structured literature review of HDTs, define the recipe to apply the paradigm, and demonstrate how higher-level observers coupled with cognitive and task-models can reveal richer insights into human engagement and learning. The work highlights practical implications for human-centric automation in industry and robotics and calls for extending the paradigm to more edge cases and richer models.

Abstract

Modern modelling approaches fail when it comes to understanding rather than pure supervision of human behavior. As humans become more and more integrated into human-to-anything interactions, the understanding of the human as a whole becomes critical. In this paper, we conduct a structured review of the human digital twin to indicate where modern paradigms fail to model the human agent. Particularly, the mechanistic viewpoint limits the usability of human and general digital twins. Instead, we propose a novel way of thinking about models, states, and their relations: Perspectives-Observer-Transparency. The modelling paradigm indicates how transparency - or whiteness - relates to the abilities of an observer, which again allows to model the penetration depth of a system model into the human psyche. The split in between the human's outer and inner states is described with a perspectives model, featuring the introperspective and the exteroperspective. We explore this novel paradigm by employing two recent scenarios from ongoing research and give examples to emphasize specific characteristics of the modelling paradigm.

Figures (6)

• Figure 1: Yearly number of publications included as per described criteria between 2018 and March 2024 (*).
• Figure 2: Two exemplary scenarios in manufacturing featuring a human-integrated system. It becomes obvious in both scenarios that human and machine are not fully integrated but merely co-exist.
• Figure 3: UML diagram of the classes in the Perspectives-Observer-Transparency modeling paradigm.
• Figure 4: Perspectives-Observer-Transparency model with an exemplary system model based on Austin et al. Austin.2020 and Secher, Seifert, and van Lieshout Secher.2008 to analyze work engagement based on diverse intermediate states. Rounded boxes are direct measures, cut boxes are states, and normal boxes are models; arrows indicate flow of information; L: Level. The oxygen-level is only the interpretation of the blood color which is derived from the color channels of an RGB image ($a$, $b$, $c$). The same is true in an IMU sensor used for detecting body movement ($d$, $e$, $f$). Both information are used to detect physiological fatigue in the exteroperspective, which functions as an indicator for psychological fatigue in the introperspective. The bridge model between them is essentially an L2 observer ($i$) embedded in L3 observer $g$. The introperspective psychological fatigue is then used as an indicator in a motivation model to evaluate work engagement as part of L4 observer $h$.
• Figure 5: Adaptation of the Knowledge Stairway for the Perspectives-Observer-Transparency modelling paradigm. The dashed line represents where experimentability is required for higher proficiency, which coincides with the border between introperspective and exteroperspective.

Theorems & Definitions (8)

• Definition 1
• Definition 2
• Definition 3
• Definition 4
• Definition 5
• Definition 6
• Definition 7
• Definition 8