Demonstration-based learning for few-shot biomedical named entity recognition under machine reading comprehension

TL;DR

In the realm of few-shot learning BioNER, MRC-based language models are much more proficient in recognizing biomedical entities compared to the sequence labeling approach and can compete successfully with fully-supervised learning methodologies that rely heavily on the availability of abundant annotated data.

Abstract

Although deep learning techniques have shown significant achievements, they frequently depend on extensive amounts of hand-labeled data and tend to perform inadequately in few-shot scenarios. The objective of this study is to devise a strategy that can improve the model's capability to recognize biomedical entities in scenarios of few-shot learning. By redefining biomedical named entity recognition (BioNER) as a machine reading comprehension (MRC) problem, we propose a demonstration-based learning method to address few-shot BioNER, which involves constructing appropriate task demonstrations. In assessing our proposed method, we compared the proposed method with existing advanced methods using six benchmark datasets, including BC4CHEMD, BC5CDR-Chemical, BC5CDR-Disease, NCBI-Disease, BC2GM, and JNLPBA. We examined the models' efficacy by reporting F1 scores from both the 25-shot and 50-shot learning experiments. In 25-shot learning, we observed 1.1% improvements in the average F1 scores compared to the baseline method, reaching 61.7%, 84.1%, 69.1%, 70.1%, 50.6%, and 59.9% on six datasets, respectively. In 50-shot learning, we further improved the average F1 scores by 1.0% compared to the baseline method, reaching 73.1%, 86.8%, 76.1%, 75.6%, 61.7%, and 65.4%, respectively. We reported that in the realm of few-shot learning BioNER, MRC-based language models are much more proficient in recognizing biomedical entities compared to the sequence labeling approach. Furthermore, our MRC-language models can compete successfully with fully-supervised learning methodologies that rely heavily on the availability of abundant annotated data. These results highlight possible pathways for future advancements in few-shot BioNER methodologies.

Figures (6)

• Figure 1: Examples to illustrate the differences between prompt-based learning, demonstration-based learning under sequence labeling, and demonstration-based learning under MRC. (a) Prompt-based learning methods often overlook entity span detection, leading to a lengthy process of generating prompts for all entity candidates. (b) Demonstration-based learning under sequence labeling can truncate the demonstration sequence during training and prediction. (c) Demonstration-based learning under MRC can effectively improve few-shot BioNER performance, provided that suitable task demonstrations are used.
• Figure 2: Performance of BERT-MRC$_{grape}$ in 25-shot, 50-shot, 100-shot, 500-shot, and fully supervised (9999) learning. The values are the average F1 scores over five runs.
• Figure 3: The impact of different batch sizes on the performance of BERT-MRC$_{grape}$ in few-shot learning. The values are the average of F1 scores over five runs.
• Figure 4: The impact of different demonstrations on the performance of BERT-MRC in few-shot learning. The values are the average of F1 scores over five runs.
• Figure 5: Implementing the BioNER task using GPT-4.