A Comprehensive Review of Sign Language Recognition: Different Types, Modalities, and Datasets

TL;DR

This paper addresses the need for a comprehensive understanding of Sign Language Recognition (SLR) across modalities, datasets, and architectures. It systematically surveys vision- and sensor-based approaches, manual and non-manual cues, isolated and continuous settings, and state-of-the-art models, highlighting data-related challenges and gaps. The analysis clarifies how dataset limitations, background variability, and signer dependence constrain generalization and real-time deployment. The authors synthesize current progress and propose directions such as multi-modal fusion, uncontrolled-environment data collection, and lightweight architectures to enable robust, signer-independent SLR in real-world human-computer interaction. The work serves as a practical guide for researchers and developers aiming to advance SLR systems for broader accessibility.

Abstract

A machine can understand human activities, and the meaning of signs can help overcome the communication barriers between the inaudible and ordinary people. Sign Language Recognition (SLR) is a fascinating research area and a crucial task concerning computer vision and pattern recognition. Recently, SLR usage has increased in many applications, but the environment, background image resolution, modalities, and datasets affect the performance a lot. Many researchers have been striving to carry out generic real-time SLR models. This review paper facilitates a comprehensive overview of SLR and discusses the needs, challenges, and problems associated with SLR. We study related works about manual and non-manual, various modalities, and datasets. Research progress and existing state-of-the-art SLR models over the past decade have been reviewed. Finally, we find the research gap and limitations in this domain and suggest future directions. This review paper will be helpful for readers and researchers to get complete guidance about SLR and the progressive design of the state-of-the-art SLR model

Figures (12)

• Figure 1: SLR Taxonomy: the fundamental attributes of SLR like datasets, modality, features, classification, computation resources, and application, along with each attribute's categorization are shown here. Large-scale datasets and modalities affect the recognition performance. The efficient features extraction method and classification model with efficient power computation resources lead to high performance.
• Figure 2: (a) SLR (b) General process flow of SLR. Figure (a) illustrates the native signer performing sign conversion into text with the help of a human-computer interface, and figure (b) illustrates the procedural stages of SLR. The recognition rate highly depends on the data set, preprocessing, feature extraction, and classifier.
• Figure 3: SLR Types: Vision-based SLR and sensor-based SLR are the SLR types. It is further, classified into manual and non-manual, then isolated and continuous.
• Figure 4: Various methods are used to extract the significant features.
• Figure 5: Manual Components. The important manual features related to sign language are shown here.