Leveraging Semantic Type Dependencies for Clinical Named Entity Recognition

TL;DR

This work tackles clinical named entity recognition by injecting domain-specific semantic type dependencies into neural models. It introduces short and long distance dependency representations derived from UMLS concepts and their semantic types, encoded as relation embeddings that connect tokens within a sentence. The approach is instantiated in BiLSTM-CRF and BiLSTM-GCN-CRF architectures and evaluated on ShARe/CLEF 2013 and i2b2/VA 2010 with embeddings including UmlsBERT, BioBERT, and BERT, showing improved precision and overall F1 in several settings. The study demonstrates that multi-type dependency information can reduce false positives and improve NER performance in clinical notes, offering a cost-effective alternative to multi-task learning and paving the way for richer domain knowledge integration in clinical NLP.

Abstract

Previous work on clinical relation extraction from free-text sentences leveraged information about semantic types from clinical knowledge bases as a part of entity representations. In this paper, we exploit additional evidence by also making use of domain-specific semantic type dependencies. We encode the relation between a span of tokens matching a Unified Medical Language System (UMLS) concept and other tokens in the sentence. We implement our method and compare against different named entity recognition (NER) architectures (i.e., BiLSTM-CRF and BiLSTM-GCN-CRF) using different pre-trained clinical embeddings (i.e., BERT, BioBERT, UMLSBert). Our experimental results on clinical datasets show that in some cases NER effectiveness can be significantly improved by making use of domain-specific semantic type dependencies. Our work is also the first study generating a matrix encoding to make use of more than three dependencies in one pass for the NER task.

Figures (2)

• Figure 1: An example of a sentence annotated with named entities, short dependencies and long dependencies with clinical knowledge exploitation. There are 3 types of entities and two types of dependencies in this example. The entity type 'disorder/problem' is highlighted using red, 'test' with green, and 'treatment' with blue. Long dependencies are relations between tokens not in the same clinical concept, which are treats (between a 'disorder/problem' entity and a 'treatment' entity) and diagnoses (between a 'treatment/disorder' entity and a 'test' entity). Examples of a treats dependency are ("cardiac disease", management"), ("dyspnea, "management"), ("chest tightness", management"), and ("symptoms, "management"). Examples of a diagnoses dependency are ("cardiac disease, "evaluation), ("dyspnea, "evaluation), ("chest tightness, "evaluation), and ("symptoms", "evaluation"). Short dependencies are relations between tokens in clinical concepts, i.e., ("cardiac", "disease") and ("chest", "tightness").
• Figure 2: A NER system architecture utilizing multiple relationships between one token and others in a sequence. The blue components represent the word embedding of multiple tokens having a relationship with a token $x_i$ at position $i$ in a text sequence. Using the example in Figure \ref{['fig:dependency_example']}: the token "disease" in the concept "cardiac disease" is the red component in this architecture and other tokens such as "evaluation" and "management" are the blue components which are concatenated with token "disease" through different relation types such as "treats" and "diagnoses". Finally, the combination of word embeddings from these relationships is concatenated with the label encoding of the semantic type for the token at position $i$, which is the green component.