Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

BioUNER: A Benchmark Dataset for Clinical Urdu Named Entity Recognition

Wazir Ali, Adeeb Noor, Sanaullah Mahar, Alia, Muhammad Mazhar Younas

Abstract

In this article, we present a gold-standard benchmark dataset for Biomedical Urdu Named Entity Recognition (BioUNER), developed by crawling health-related articles from online Urdu news portals, medical prescriptions, and hospital health blogs and websites. After preprocessing, three native annotators with familiarity in the medical domain participated in the annotation process using the Doccano text annotation tool and annotated 153K tokens. Following annotation, the proposed BioiUNER dataset was evaluated both intrinsically and extrinsically. An inter-annotator agreement score of 0.78 was achieved, thereby validating the dataset as gold-standard quality. To demonstrate the utility and benchmarking capability of the dataset, we evaluated several machine learning and deep learning models, including Support Vector Machines (SVM), Long Short-Term Memory networks (LSTM), Multilingual BERT (mBERT), and XLM-RoBERTa. The gold-standard BioUNER dataset serves as a reliable benchmark and a valuable addition to Urdu language processing resources.

BioUNER: A Benchmark Dataset for Clinical Urdu Named Entity Recognition

Abstract

In this article, we present a gold-standard benchmark dataset for Biomedical Urdu Named Entity Recognition (BioUNER), developed by crawling health-related articles from online Urdu news portals, medical prescriptions, and hospital health blogs and websites. After preprocessing, three native annotators with familiarity in the medical domain participated in the annotation process using the Doccano text annotation tool and annotated 153K tokens. Following annotation, the proposed BioiUNER dataset was evaluated both intrinsically and extrinsically. An inter-annotator agreement score of 0.78 was achieved, thereby validating the dataset as gold-standard quality. To demonstrate the utility and benchmarking capability of the dataset, we evaluated several machine learning and deep learning models, including Support Vector Machines (SVM), Long Short-Term Memory networks (LSTM), Multilingual BERT (mBERT), and XLM-RoBERTa. The gold-standard BioUNER dataset serves as a reliable benchmark and a valuable addition to Urdu language processing resources.

Paper Structure

This paper contains 23 sections, 9 equations, 1 figure, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Data Collection & Annotation
  4. Data Collection
  5. Preprocessing
  6. Annotation
  7. Quality Assessment
  8. BioUNER Dataset
  9. Experimental Setup
  10. Task Definition
  11. Experimental Setup
  12. Task Definition and Dataset
  13. Baseline Models
  14. Support Vector Machine (SVM)
  15. CRF
  16. ...and 8 more sections

Figures (1)

  • Figure 1: An example of annotation of BioUNER using Doccano text annotation tool