Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Medical Coding with Biomedical Transformer Ensembles and Zero/Few-shot Learning

Angelo Ziletti, Alan Akbik, Christoph Berns, Thomas Herold, Marion Legler, Martina Viell

TL;DR

The paper tackles medical coding (MC) in MedDRA with an extremely large label space of about $80{,}000$ LLTs and long-tail data. It proposes xTARS, a hybrid approach that combines a BERT-based multiclass classifier with a zero/few-shot learner, using top-$n$ candidate labels and enhanced negative sampling to scale to large label sets. Experiments show xTARS outperforms strong baselines, especially in few-shot/uncertain cases, and the system is deployed at Bayer with promising real-world performance (LLT accuracy ~90% at 80% coverage). The authors release their code to the research community, highlighting practical impact for automated medical coding and potential extensions beyond MC.

Abstract

Medical coding (MC) is an essential pre-requisite for reliable data retrieval and reporting. Given a free-text reported term (RT) such as "pain of right thigh to the knee", the task is to identify the matching lowest-level term (LLT) - in this case "unilateral leg pain" - from a very large and continuously growing repository of standardized medical terms. However, automating this task is challenging due to a large number of LLT codes (as of writing over 80,000), limited availability of training data for long tail/emerging classes, and the general high accuracy demands of the medical domain. With this paper, we introduce the MC task, discuss its challenges, and present a novel approach called xTARS that combines traditional BERT-based classification with a recent zero/few-shot learning approach (TARS). We present extensive experiments that show that our combined approach outperforms strong baselines, especially in the few-shot regime. The approach is developed and deployed at Bayer, live since November 2021. As we believe our approach potentially promising beyond MC, and to ensure reproducibility, we release the code to the research community.

Medical Coding with Biomedical Transformer Ensembles and Zero/Few-shot Learning

TL;DR

The paper tackles medical coding (MC) in MedDRA with an extremely large label space of about LLTs and long-tail data. It proposes xTARS, a hybrid approach that combines a BERT-based multiclass classifier with a zero/few-shot learner, using top- candidate labels and enhanced negative sampling to scale to large label sets. Experiments show xTARS outperforms strong baselines, especially in few-shot/uncertain cases, and the system is deployed at Bayer with promising real-world performance (LLT accuracy ~90% at 80% coverage). The authors release their code to the research community, highlighting practical impact for automated medical coding and potential extensions beyond MC.

Abstract

Medical coding (MC) is an essential pre-requisite for reliable data retrieval and reporting. Given a free-text reported term (RT) such as "pain of right thigh to the knee", the task is to identify the matching lowest-level term (LLT) - in this case "unilateral leg pain" - from a very large and continuously growing repository of standardized medical terms. However, automating this task is challenging due to a large number of LLT codes (as of writing over 80,000), limited availability of training data for long tail/emerging classes, and the general high accuracy demands of the medical domain. With this paper, we introduce the MC task, discuss its challenges, and present a novel approach called xTARS that combines traditional BERT-based classification with a recent zero/few-shot learning approach (TARS). We present extensive experiments that show that our combined approach outperforms strong baselines, especially in the few-shot regime. The approach is developed and deployed at Bayer, live since November 2021. As we believe our approach potentially promising beyond MC, and to ensure reproducibility, we release the code to the research community.
Paper Structure (31 sections, 4 figures, 4 tables)

This paper contains 31 sections, 4 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Task and Data Sources
  3. MedDRA as target ontology
  4. Training data
  5. Method
  6. Baseline 1: Multiclass Classification with BERT Ensembles
  7. Baseline 2: TARS Few-Shot Classification
  8. Proposed Approach: xTARS
  9. Evaluation
  10. Experimental setup
  11. Experimental results
  12. Discussion and practical deployment
  13. Related work
  14. Conclusions
  15. Appendix
  16. ...and 16 more sections

Figures (4)

  • Figure 1: From raw data sources to training, validation, and test data via data processing and data augmentation (BERT models).
  • Figure 2: From raw data sources to training, validation, and test data via data processing and data augmentation (TARS and xTARS models). Data coming from the ontology is omitted for computational reasons.
  • Figure 3: Cloud architecture of the deployed medical coding system outlined in the main text.
  • Figure 4: Screenshot of the medical coding platform where the coding solutions proposed by the algorithm described in the main text are shown to medical coders for acceptance or rejection.