Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Multilingual Clinical NER for Diseases and Medications Recognition in Cardiology Texts using BERT Embeddings

Manuela Daniela Danu, George Marica, Constantin Suciu, Lucian Mihai Itu, Oladimeji Farri

TL;DR

This study tackles multilingual clinical NER for diseases and medications in cardiology texts by fine-tuning general-domain BERT models on domain-specific bilingual and trilingual data. It deploys four monolingual and two multilingual models across Spanish, English, and Italian within the BioASQ MultiCardioNER framework, evaluating two subtasks: Spanish diseases recognition and multilingual medications recognition. The results show strong gains in medication recognition across languages, with SDR, SMR, EMR, and IMR achieving high F1 scores and outperforming leaderboard baselines, though disease recognition remains more challenging and exhibits some overfitting in fine-tuned variants. The work highlights the potential of transfer learning from general-domain to biomedical multilingual NER and paves the way for integrating larger language models to further enhance clinical information extraction in cardiology.

Abstract

The rapidly increasing volume of electronic health record (EHR) data underscores a pressing need to unlock biomedical knowledge from unstructured clinical texts to support advancements in data-driven clinical systems, including patient diagnosis, disease progression monitoring, treatment effects assessment, prediction of future clinical events, etc. While contextualized language models have demonstrated impressive performance improvements for named entity recognition (NER) systems in English corpora, there remains a scarcity of research focused on clinical texts in low-resource languages. To bridge this gap, our study aims to develop multiple deep contextual embedding models to enhance clinical NER in the cardiology domain, as part of the BioASQ MultiCardioNER shared task. We explore the effectiveness of different monolingual and multilingual BERT-based models, trained on general domain text, for extracting disease and medication mentions from clinical case reports written in English, Spanish, and Italian. We achieved an F1-score of 77.88% on Spanish Diseases Recognition (SDR), 92.09% on Spanish Medications Recognition (SMR), 91.74% on English Medications Recognition (EMR), and 88.9% on Italian Medications Recognition (IMR). These results outperform the mean and median F1 scores in the test leaderboard across all subtasks, with the mean/median values being: 69.61%/75.66% for SDR, 81.22%/90.18% for SMR, 89.2%/88.96% for EMR, and 82.8%/87.76% for IMR.

Multilingual Clinical NER for Diseases and Medications Recognition in Cardiology Texts using BERT Embeddings

TL;DR

This study tackles multilingual clinical NER for diseases and medications in cardiology texts by fine-tuning general-domain BERT models on domain-specific bilingual and trilingual data. It deploys four monolingual and two multilingual models across Spanish, English, and Italian within the BioASQ MultiCardioNER framework, evaluating two subtasks: Spanish diseases recognition and multilingual medications recognition. The results show strong gains in medication recognition across languages, with SDR, SMR, EMR, and IMR achieving high F1 scores and outperforming leaderboard baselines, though disease recognition remains more challenging and exhibits some overfitting in fine-tuned variants. The work highlights the potential of transfer learning from general-domain to biomedical multilingual NER and paves the way for integrating larger language models to further enhance clinical information extraction in cardiology.

Abstract

The rapidly increasing volume of electronic health record (EHR) data underscores a pressing need to unlock biomedical knowledge from unstructured clinical texts to support advancements in data-driven clinical systems, including patient diagnosis, disease progression monitoring, treatment effects assessment, prediction of future clinical events, etc. While contextualized language models have demonstrated impressive performance improvements for named entity recognition (NER) systems in English corpora, there remains a scarcity of research focused on clinical texts in low-resource languages. To bridge this gap, our study aims to develop multiple deep contextual embedding models to enhance clinical NER in the cardiology domain, as part of the BioASQ MultiCardioNER shared task. We explore the effectiveness of different monolingual and multilingual BERT-based models, trained on general domain text, for extracting disease and medication mentions from clinical case reports written in English, Spanish, and Italian. We achieved an F1-score of 77.88% on Spanish Diseases Recognition (SDR), 92.09% on Spanish Medications Recognition (SMR), 91.74% on English Medications Recognition (EMR), and 88.9% on Italian Medications Recognition (IMR). These results outperform the mean and median F1 scores in the test leaderboard across all subtasks, with the mean/median values being: 69.61%/75.66% for SDR, 81.22%/90.18% for SMR, 89.2%/88.96% for EMR, and 82.8%/87.76% for IMR.
Paper Structure (9 sections, 5 figures, 1 table)

This paper contains 9 sections, 5 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methods
  4. Datasets
  5. Experiments
  6. Subtask 1: Diseases Recognition in Spanish Cardiology Texts
  7. Subtask 2: Multilingual (Spanish, English and Italian) Medications Recognition in Cardiology Texts
  8. Results
  9. Conclusions

Figures (5)

  • Figure 1: Overview of the prediction pipeline. As a pre-processing step, the clinical case reports are split into sentences, and further segmented into word-level tokens. By leveraging the available BRAT annotations, the word-level tokens are encoded using the BIO format and used to fine-tune BERT-based models on the MultiCardioNER dataset. The output from the BERT models, also in BIO format, is then post-processed to comply with BRAT format.
  • Figure 2: Prediction example for the Spanish Diseases Recognition (SDR) subtask, obtained using the best performing model in terms of F1-score. Green represents correctly identified mentions along with their spans. Red represents mentions that are not present in the ground-truth but predicted by the model. Yellow refers to mentions that are incompletely predicted by the model, while orange marks the full mention as present in the ground-truth.
  • Figure 3: Prediction example for the Spanish Medications Recognition (SMR) subtask, obtained using the best performing model in terms of F1-score. Green represents correctly identified mentions along with their spans. In this particular example, there were no missed, incomplete, or incorrect predictions.
  • Figure 4: Prediction example for the English Medications Recognition (EMR) subtask, obtained using the best performing model in terms of F1-score. Green represents correctly identified mentions along with their spans. In this particular example, there were no missed, incomplete, or incorrect predictions.
  • Figure 5: Prediction example for the Italian Medications Recognition (IMR) subtask, obtained using the best performing model in terms of F1-score. Green represents correctly identified mentions along with their spans. In this particular example, there were no missed, incomplete, or incorrect predictions .