Entity Decomposition with Filtering: A Zero-Shot Clinical Named Entity Recognition Framework

TL;DR

This work tackles zero-shot clinical NER by leveraging open NER LLMs through a novel EDF framework. EDF decomposes a target entity type into sub-types, uses an open NER LLM to retrieve sub-type entities, and applies context-aware filtering to produce accurate final extractions, enabling robust performance without training data. Across multiple public clinical datasets and entity types, EDF improves recall via decomposition and precision via filtering, with average F1 gains of $2.54\%$ (UniversalNER) and $5.82\%$ (GNER). The paper also presents thorough ablations and error analyses, showing the framework’s robustness to decomposer and filter choices and highlighting remaining challenges such as abbreviations and polarity-driven context effects. Overall, EDF offers a cost-efficient, zero-shot pathway for clinical NER with open models, guiding future expansion to additional entity types and datasets.

Abstract

Clinical named entity recognition (NER) aims to retrieve important entities within clinical narratives. Recent works have demonstrated that large language models (LLMs) can achieve strong performance in this task. While previous works focus on proprietary LLMs, we investigate how open NER LLMs, trained specifically for entity recognition, perform in clinical NER. Our initial experiment reveals significant contrast in performance for some clinical entities and how a simple exploitment on entity types can alleviate this issue. In this paper, we introduce a novel framework, entity decomposition with filtering, or EDF. Our key idea is to decompose the entity recognition task into several retrievals of entity sub-types and then filter them. Our experimental results demonstrate the efficacies of our framework and the improvements across all metrics, models, datasets, and entity types. Our analysis also reveals substantial improvement in recognizing previously missed entities using entity decomposition. We further provide a comprehensive evaluation of our framework and an in-depth error analysis to pave future works.

Figures (7)

• Figure 1: Entity Decomposition with Filtering. Our novel framework is composed of three components: (1) Entity Decomposer breaks down the target entity type (e.g., treatment) into several entity sub-types (e.g., medication, medical device, medical procedure, etc), (2) Open NER LLM generates the sub-type entities, (3) Filter removes sub-type entities outside the target entity type. See \ref{['sec_method']} for details.
• Figure 2: Open NER LLM (UniversalNER) performs better at extracting entity sub-type (Med) rather than the target entity type (Tr, Pr, Te). We use ClinicalIE for Med (medication) and i2b2 2012 for Tr (treatment), Pr (problem), and Te (test). Discussion in \ref{['subsec_prelim']}
• Figure 3: Average performance improvement to baseline across metrics and models. Entity decomposition (ED) improves recall but degrades precision. Filter (F) increases precision but decreases recall. Our method (EDF) achieves better performance overall. Discussion in \ref{['subsec_overall_perf']}
• Figure 4: Entity Decomposition (ED) captures most previously fully absent entities. Lower value is better. Discussion in \ref{['subsubsec_error1']}
• Figure 5: The majority of the rejected golden "Problem" entities are negative, leading to a performance drop in \ref{['table_filter_context']} when using context. Further discussion in \ref{['subsubsec_error4']}.