Complying with the EU AI Act: Innovations in Explainable and User-Centric Hand Gesture Recognition

TL;DR

This work presents XentricAI, an explainable, user-centric hand gesture recognition system built for EU AI Act alignment. It couples a GRU-based HGR model with transfer learning via experience replay, a tailored VAE for anomaly detection with user-specific thresholds, and SHAP-based anomalous gesture characterization to enhance transparency. Key results include 11.5% more detectable anomalous gestures and 97.5% accuracy in characterizing anomalies, plus at least a 15.17% improvement in user adaptability through personalization. The approach demonstrates strong regulatory relevance by addressing transparency, human oversight, and data governance, while expanding a real-world FMCW radar gesture dataset with diverse, out-of-distribution examples. Overall, XentricAI offers a practical, scalable path to trustworthy, commercially viable HGR systems under the EU AI Act.

Abstract

The EU AI Act underscores the importance of transparency, user-centricity, and robustness in AI systems, particularly for high-risk systems. In response, we present advancements in XentricAI, an explainable hand gesture recognition (HGR) system designed to meet these regulatory requirements. XentricAI adresses fundamental challenges in HGR, such as the opacity of black-box models using explainable AI methods and the handling of distributional shifts in real-world data through transfer learning techniques. We extend an existing radar-based HGR dataset by adding 28,000 new gestures, with contributions from multiple users across varied locations, including 24,000 out-of-distribution gestures. Leveraging this real-world dataset, we enhance XentricAI's capabilities by integrating a variational autoencoder module for improved gesture anomaly detection, incorporating user-specific thresholding. This integration enables the identification of 11.50% more anomalous gestures. Our extensive evaluations demonstrate a 97.5% sucess rate in characterizing these anomalies, significantly improving system explainability. Furthermore, the implementation of transfer learning techniques has shown a substantial increase in user adaptability, with an average improvement of at least 15.17%. This work contributes to the development of trustworthy AI systems by providing both technical advancements and regulatory compliance, offering a commercially viable solution that aligns with the EU AI Act requirements.

Figures (15)

• Figure 1: Overview of the EU AI Act's framework. A) Pyramid illustrating the risk-based approach of the EU AI Act. B) Ethical principles and requirements for trustworthy AI as outlined in the EU AI Act.
• Figure 2: Comparison of AI models. Explainable AI involves opening the black-box to identify the factors that influenced the model's decisions, whereas interpretable AI uses a model that is inherently understandable and transparent in its decision-making process.
• Figure 3: Overview of the XentricAI methodology. This comprehensive framework for HGR and analysis is composed of four interconnected building blocks
• Figure 4: Exemplary illustration of dataset allocation for model training, calibration, and assessment. (A) Training and validation datasets are derived from most data from $user_1$ and $user_2$, with a small portion set aside for forgetting rate assessment. (B) Data from the remaining users is divided between model calibration without ER and calibration assessment. (C) For model calibration with ER, $n_\text{train}$-$n_{\text{user,}i}$ amount of user and training data is added to the retraining dataset, creating the ER-augmented retraining dataset.
• Figure 5: Windowing and preprocessing of gesture sequences for SHAP analysis.