Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Malayalam Sign Language Identification using Finetuned YOLOv8 and Computer Vision Techniques

Abhinand K., Abhiram B. Nair, Dhananjay C., Hanan Hamza, Mohammed Fawaz J., Rahma Fahim K., Anoop V. S

TL;DR

The paper tackles automatic recognition of Malayalam Sign Language (MSL) gestures to improve communication for hearing-impaired individuals in Kerala. It builds a dedicated 20-sign MSL dataset, employs data augmentation, and trains a YOLOv8-based detector with transfer learning, achieving an mAP of about 83.9% on validation. The workflow leverages Roboflow for data management and Ultralytics YOLOv8 for end-to-end training, demonstrating a robust baseline for Malayalam sign identification. The work highlights near real-time performance and discusses extending the system to dynamic signs with potential impact in education, emergency communication, and inclusive technologies.

Abstract

Technological advancements and innovations are advancing our daily life in all the ways possible but there is a larger section of society who are deprived of accessing the benefits due to their physical inabilities. To reap the real benefits and make it accessible to society, these talented and gifted people should also use such innovations without any hurdles. Many applications developed these days address these challenges, but localized communities and other constrained linguistic groups may find it difficult to use them. Malayalam, a Dravidian language spoken in the Indian state of Kerala is one of the twenty-two scheduled languages in India. Recent years have witnessed a surge in the development of systems and tools in Malayalam, addressing the needs of Kerala, but many of them are not empathetically designed to cater to the needs of hearing-impaired people. One of the major challenges is the limited or no availability of sign language data for the Malayalam language and sufficient efforts are not made in this direction. In this connection, this paper proposes an approach for sign language identification for the Malayalam language using advanced deep learning and computer vision techniques. We start by developing a labeled dataset for Malayalam letters and for the identification we use advanced deep learning techniques such as YOLOv8 and computer vision. Experimental results show that the identification accuracy is comparable to other sign language identification systems and other researchers in sign language identification can use the model as a baseline to develop advanced models.

Malayalam Sign Language Identification using Finetuned YOLOv8 and Computer Vision Techniques

TL;DR

The paper tackles automatic recognition of Malayalam Sign Language (MSL) gestures to improve communication for hearing-impaired individuals in Kerala. It builds a dedicated 20-sign MSL dataset, employs data augmentation, and trains a YOLOv8-based detector with transfer learning, achieving an mAP of about 83.9% on validation. The workflow leverages Roboflow for data management and Ultralytics YOLOv8 for end-to-end training, demonstrating a robust baseline for Malayalam sign identification. The work highlights near real-time performance and discusses extending the system to dynamic signs with potential impact in education, emergency communication, and inclusive technologies.

Abstract

Technological advancements and innovations are advancing our daily life in all the ways possible but there is a larger section of society who are deprived of accessing the benefits due to their physical inabilities. To reap the real benefits and make it accessible to society, these talented and gifted people should also use such innovations without any hurdles. Many applications developed these days address these challenges, but localized communities and other constrained linguistic groups may find it difficult to use them. Malayalam, a Dravidian language spoken in the Indian state of Kerala is one of the twenty-two scheduled languages in India. Recent years have witnessed a surge in the development of systems and tools in Malayalam, addressing the needs of Kerala, but many of them are not empathetically designed to cater to the needs of hearing-impaired people. One of the major challenges is the limited or no availability of sign language data for the Malayalam language and sufficient efforts are not made in this direction. In this connection, this paper proposes an approach for sign language identification for the Malayalam language using advanced deep learning and computer vision techniques. We start by developing a labeled dataset for Malayalam letters and for the identification we use advanced deep learning techniques such as YOLOv8 and computer vision. Experimental results show that the identification accuracy is comparable to other sign language identification systems and other researchers in sign language identification can use the model as a baseline to develop advanced models.
Paper Structure (12 sections, 8 figures)

This paper contains 12 sections, 8 figures.

Table of Contents

  1. Introduction
  2. Materials and Methods
  3. Roboflow
  4. Ultralytics
  5. YOLOv8
  6. Dataset
  7. Proposed Methodology
  8. Experiments
  9. Data Augmentation
  10. Load and build Ultralytics YOLOv8 model
  11. Results and Discussions
  12. Conclusions and Future Work

Figures (8)

  • Figure 1: Examples of 6 Malayalam Sign Language (MSL) characters
  • Figure 2: A diagrammatic representation of YOLOv8 architecture
  • Figure 3: Sample images from the dataset created
  • Figure 4: The overall workflow of sign language detection using pre-trained model
  • Figure 5: Confusion matrix of the model
  • ...and 3 more figures