Automatic Speech Recognition for Sanskrit with Transfer Learning
Bidit Sadhukhan, Swami Punyeshwarananda
TL;DR
The paper tackles the low-resource problem of Sanskrit ASR by transferring learning from the multilingual Whisper model. By fine-tuning Whisper Small and Medium on the Vāksañcayaḥ dataset and optimizing hyper-parameters, the authors achieve a WER of 15.42% on the test set and demonstrate robustness on an out-of-domain split. The approach leverages log-Mel spectrogram preprocessing, encoder–decoder transformers, and Hugging Face tooling to build effective Sanskrit transcription with an accessible online demo. This work advances Sanskrit accessibility for transcription, education, and research, and suggests directions such as larger datasets, Sanskrit-specific tokenization, and multimodal extensions for future improvements.
Abstract
Sanskrit, one of humanity's most ancient languages, has a vast collection of books and manuscripts on diverse topics that have been accumulated over millennia. However, its digital content (audio and text), which is vital for the training of AI systems, is profoundly limited. Furthermore, its intricate linguistics make it hard to develop robust NLP tools for wider accessibility. Given these constraints, we have developed an automatic speech recognition model for Sanskrit by employing transfer learning mechanism on OpenAI's Whisper model. After carefully optimising the hyper-parameters, we obtained promising results with our transfer-learned model achieving a word error rate of 15.42% on Vaksancayah dataset. An online demo of our model is made available for the use of public and to evaluate its performance firsthand thereby paving the way for improved accessibility and technological support for Sanskrit learning in the modern era.