Bangla MedER: Multi-BERT Ensemble Approach for the Recognition of Bangla Medical Entity

TL;DR

Bangla MedER addresses the scarcity of annotated Bangla medical data and the challenge of domain-specific terminology in medical NER. The authors introduce Bangla MedER and a Multi-BERT Ensemble that combines two BERT-based layers, each processing a different input sequence, with their representations concatenated for final classification. On a newly created Bangla medical dataset (6,895 statements) spanning six entity types, the ensemble achieves 89.58% accuracy and a macro F1 of 87.87%, outperforming single-layer BERT by 11.8 percentage points in accuracy. The study demonstrates the viability of transformer ensembles for Bangla medical NLP, provides publicly available data, and lays groundwork for broader domain adaptation and healthcare applications in Bangla-speaking settings.

Abstract

Medical Entity Recognition (MedER) is an essential NLP task for extracting meaningful entities from the medical corpus. Nowadays, MedER-based research outcomes can remarkably contribute to the development of automated systems in the medical sector, ultimately enhancing patient care and outcomes. While extensive research has been conducted on MedER in English, low-resource languages like Bangla remain underexplored. Our work aims to bridge this gap. For Bangla medical entity recognition, this study first examined a number of transformer models, including BERT, DistilBERT, ELECTRA, and RoBERTa. We also propose a novel Multi-BERT Ensemble approach that outperformed all baseline models with the highest accuracy of 89.58%. Notably, it provides an 11.80% accuracy improvement over the single-layer BERT model, demonstrating its effectiveness for this task. A major challenge in MedER for low-resource languages is the lack of annotated datasets. To address this issue, we developed a high-quality dataset tailored for the Bangla MedER task. The dataset was used to evaluate the effectiveness of our model through multiple performance metrics, demonstrating its robustness and applicability. Our findings highlight the potential of Multi-BERT Ensemble models in improving MedER for Bangla and set the foundation for further advancements in low-resource medical NLP.

Figures (7)

• Figure 1: System architecture
• Figure 2: Indicating the number of observations per entity
• Figure 3: Data Preprocessing Process of MedER
• Figure 4: Proposed-Model
• Figure 5: Confusion Matrix of the proposed model