LRW-Persian: Lip-reading in the Wild Dataset for Persian Language

TL;DR

LRW-Persian addresses the paucity of Persian visual-speech resources by introducing LRW-Persian, a large in-the-wild word-level lip-reading corpus with $743$ target words and over $414{,}308$ clips drawn from nearly $1{,}989$ hours of TV. The authors propose an end-to-end curation pipeline combining ASR-based transcription, active-speaker localization, and perceptual filtering to ensure frontal, unoccluded faces with rich per-clip metadata. They benchmark two architectures—Multi-Scale Temporal Convolutional Network and ResNet$+$BiLSTM—on the dataset, revealing a performance gap relative to English lip-reading benchmarks and underscoring the difficulty of Persian visual speech recognition. The resource enables rigorous benchmarking, cross-lingual transfer, and multimodal research for underrepresented languages, and is publicly available.

Abstract

Lipreading has emerged as an increasingly important research area for developing robust speech recognition systems and assistive technologies for the hearing-impaired. However, non-English resources for visual speech recognition remain limited. We introduce LRW-Persian, the largest in-the-wild Persian word-level lipreading dataset, comprising $743$ target words and over $414{,}000$ video samples extracted from more than $1{,}900$ hours of footage across $67$ television programs. Designed as a benchmark-ready resource, LRW-Persian provides speaker-disjoint training and test splits, wide regional and dialectal coverage, and rich per-clip metadata including head pose, age, and gender. To ensure large-scale data quality, we establish a fully automated end-to-end curation pipeline encompassing transcription based on Automatic Speech Recognition(ASR), active-speaker localization, quality filtering, and pose/mask screening. We further fine-tune two widely used lipreading architectures on LRW-Persian, establishing reference performance and demonstrating the difficulty of Persian visual speech recognition. By filling a critical gap in low-resource languages, LRW-Persian enables rigorous benchmarking, supports cross-lingual transfer, and provides a foundation for advancing multimodal speech research in underrepresented linguistic contexts. The dataset is publicly available at: https://lrw-persian.vercel.app.

Figures (6)

• Figure 1: The end-to-end data construction pipeline for LRW-Persian.
• Figure 2: Example samples from the LRW-Persian dataset. Representative frames illustrating diversity in speakers, lighting conditions, and head poses across different programs.
• Figure 3: Population pyramid illustrating the age distribution of male (blue) and female (red) speakers, with horizontal bars representing the number of speakers (in thousands) across 5-year age bins.
• Figure 4: Illustration of roll, pitch, and yaw. Figure from dubey2022image.
• Figure 5: Frequency histograms of roll, pitch, and yaw angles for all the samples in the dataset.