Biometric-enabled Personalized Augmentative and Alternative Communications

TL;DR

The proposed methodology is used to assess the gaps between the social and practical demands, such as assisting people with communication disabilities in the contemporary semi-automated border control, and the emerging advances in AI, such as advanced video and speech processing.

Abstract

This study focuses on the roadmapping of biometric technologies onto personalized Augmentative and Alternative Communication (AAC), a branch of assistive technologies for people with communication disabilities. This technology roadmapping revolves around the proposed notions of an AAC biometric register and biometric-enabled reconfigurable AAC channels. The biometric register is referred to as a tool for acquiring and processing physiological and behavioural traits that are essential for augmentative and alternative communication. It links biometric traits, such as gestures, to intermediate traits, such as synthesized speech, for customizable communication channels. The proposed methodology is used to assess the gaps between the social and practical demands, such as assisting people with communication disabilities in the contemporary semi-automated border control, and the emerging advances in AI, such as advanced video and speech processing. We provide two case studies of the AAC that rely on hand gesture recognition and sign language word recognition, and conclude that the current accuracy of those AI technologies does not meet the practical requirements. The proposed roadmapping provides recommendations for further improvement to close these gaps.

Figures (14)

• Figure 1: Specification of comparative analysis. Two kinds of AAC messages for individual A: communication with individual B (left) and sensing the environment (right). Both are in demand for personalization, but we are focused only on AAC messages.
• Figure 2: The three scenarios for deployment of the personalized AAC systems: $(a)$ an autonomous system installed on a personal computer; $(b)$ a system that uses cloud resources via smartphone apps; $(c)$ a system that uses cloud resources via an on-body communication hub.
• Figure 3: Personalized AAC computing includes a human-in-the-loop principle and its implementation using a self-aware computing, the cognitive dynamic models, and the digital twin.
• Figure 4: Our approach is compatible with the most efficient model of personalization known as a human digital twin. The proposed technology roadmap includes the reference technology milestones as shown on the left plane. Causal mapping of these reference milestones onto the AAC technology landscape results in a set of milestone candidates (right plane). This technology landscape is a guide to the strategy for developers of the future generation of AAC systems.
• Figure 5: Illustration of the biometric-centric AAC personalization within the dimensions of biometric traits, biometric transformations, and their digital twins, including synthetic biometrics. The AAC biometric register that captures the 9 most usable biometric traits is the core of the personalized AAC technology roadmapping. The relationships between the biometric traits are determined by the transformations, or translations, to enable communication for individuals with disabilities.