Bengali Text Classification: An Evaluation of Large Language Model Approaches
Md Mahmudul Hoque, Md Mehedi Hassain, Md Hojaifa Tanvir, Rahul Nandy
TL;DR
This paper addresses Bengali news text classification under resource-constrained conditions by benchmarking three instruction-tuned LLMs—LLaMA 3.1 8B Instruct, LLaMA 3.2 3B Instruct, and Qwen 2.5 7B Instruct—on a Prothom Alo Bengali dataset. Using an 80/20 train-test split and a 9-category schema, the study implements data preprocessing (duplicate removal, Random Under Sampler), and model development with LoRA/QLoRA and 4-bit quantization, guided by a purpose-built prompt. Results show Qwen 2.5 achieving the top accuracy of 72%, outperforming LLaMA variants (53% and 56%), with per-category strengths in Sports, Technology, and Education and notable confusion among closely related categories. The work demonstrates the viability of LLMs for Bengali NLP and lays groundwork for future improvements through broader model exploration, improved class balance, and enhanced fine-tuning strategies.
Abstract
Bengali text classification is a Significant task in natural language processing (NLP), where text is categorized into predefined labels. Unlike English, Bengali faces challenges due to the lack of extensive annotated datasets and pre-trained language models. This study explores the effectiveness of large language models (LLMs) in classifying Bengali newspaper articles. The dataset used, obtained from Kaggle, consists of articles from Prothom Alo, a major Bangladeshi newspaper. Three instruction-tuned LLMs LLaMA 3.1 8B Instruct, LLaMA 3.2 3B Instruct, and Qwen 2.5 7B Instruct were evaluated for this task under the same classification framework. Among the evaluated models, Qwen 2.5 achieved the highest classification accuracy of 72%, showing particular strength in the "Sports" category. In comparison, LLaMA 3.1 and LLaMA 3.2 attained accuracies of 53% and 56%, respectively. The findings highlight the effectiveness of LLMs in Bengali text classification, despite the scarcity of resources for Bengali NLP. Future research will focus on exploring additional models, addressing class imbalance issues, and refining fine-tuning approaches to improve classification performance.