Real-time Bangla Sign Language Translator

TL;DR

This work targets real-time translation of Bangla sign language to written Bangla text to bridge communication gaps for the deaf community. It presents a pipeline that combines Mediapipe Holistic landmark extraction, LSTM-based sequential translation, and computer vision with Bangla font rendering, reporting an accuracy of 94% and a 93% F1 score on collected data. Data collection involved 30-frame samples per word, and the system runs in real time, demonstrating practical applicability in Bangla-speaking contexts. The study highlights significant societal impact for accessibility and outlines future directions toward sign-to-voice output and NLP integration to further enhance communication for sign language users.

Abstract

The human body communicates through various meaningful gestures, with sign language using hands being a prominent example. Bangla Sign Language Translation (BSLT) aims to bridge communication gaps for the deaf and mute community. Our approach involves using Mediapipe Holistic to gather key points, LSTM architecture for data training, and Computer Vision for realtime sign language detection with an accuracy of 94%. Keywords=Recurrent Neural Network, LSTM, Computer Vision, Bangla font.

Figures (12)

• Figure 1: Bangla alphabet sign [7]
• Figure 2: An overview of SLT approach .[1]
• Figure 3: 3D CNN and LSTM Encoding and Decoding Structure[3]
• Figure 4: An overview of CorrNet [4]
• Figure 5: The testing procedure flowchart [5]