Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

PromptLink: Leveraging Large Language Models for Cross-Source Biomedical Concept Linking

Yuzhang Xie, Jiaying Lu, Joyce Ho, Fadi Nahab, Xiao Hu, Carl Yang

TL;DR

The paper addresses the challenge of linking biomedical concepts across heterogeneous data sources where naming conventions diverge. It introduces PromptLink, a zero-shot framework that first generates candidate concepts with SapBERT embeddings and cosine similarity, then applies a two-stage prompting strategy with GPT-4—enhanced by self-consistency and self-verification—to produce reliable links and NIL decisions. Empirical results on MIID and CISE show PromptLink outperforming strong baselines, with clear NIL-prediction capabilities and cost-efficient prompting. The approach offers strong generalization across data sources without requiring training data, making it well-suited for integrating EHR data with knowledge graphs and similar biomedical resources. Future work may focus on further reducing prompting costs and extending the framework to additional biomedical data sources and topologies.

Abstract

Linking (aligning) biomedical concepts across diverse data sources enables various integrative analyses, but it is challenging due to the discrepancies in concept naming conventions. Various strategies have been developed to overcome this challenge, such as those based on string-matching rules, manually crafted thesauri, and machine learning models. However, these methods are constrained by limited prior biomedical knowledge and can hardly generalize beyond the limited amounts of rules, thesauri, or training samples. Recently, large language models (LLMs) have exhibited impressive results in diverse biomedical NLP tasks due to their unprecedentedly rich prior knowledge and strong zero-shot prediction abilities. However, LLMs suffer from issues including high costs, limited context length, and unreliable predictions. In this research, we propose PromptLink, a novel biomedical concept linking framework that leverages LLMs. It first employs a biomedical-specialized pre-trained language model to generate candidate concepts that can fit in the LLM context windows. Then it utilizes an LLM to link concepts through two-stage prompts, where the first-stage prompt aims to elicit the biomedical prior knowledge from the LLM for the concept linking task and the second-stage prompt enforces the LLM to reflect on its own predictions to further enhance their reliability. Empirical results on the concept linking task between two EHR datasets and an external biomedical KG demonstrate the effectiveness of PromptLink. Furthermore, PromptLink is a generic framework without reliance on additional prior knowledge, context, or training data, making it well-suited for concept linking across various types of data sources. The source code is available at https://github.com/constantjxyz/PromptLink.

PromptLink: Leveraging Large Language Models for Cross-Source Biomedical Concept Linking

TL;DR

The paper addresses the challenge of linking biomedical concepts across heterogeneous data sources where naming conventions diverge. It introduces PromptLink, a zero-shot framework that first generates candidate concepts with SapBERT embeddings and cosine similarity, then applies a two-stage prompting strategy with GPT-4—enhanced by self-consistency and self-verification—to produce reliable links and NIL decisions. Empirical results on MIID and CISE show PromptLink outperforming strong baselines, with clear NIL-prediction capabilities and cost-efficient prompting. The approach offers strong generalization across data sources without requiring training data, making it well-suited for integrating EHR data with knowledge graphs and similar biomedical resources. Future work may focus on further reducing prompting costs and extending the framework to additional biomedical data sources and topologies.

Abstract

Linking (aligning) biomedical concepts across diverse data sources enables various integrative analyses, but it is challenging due to the discrepancies in concept naming conventions. Various strategies have been developed to overcome this challenge, such as those based on string-matching rules, manually crafted thesauri, and machine learning models. However, these methods are constrained by limited prior biomedical knowledge and can hardly generalize beyond the limited amounts of rules, thesauri, or training samples. Recently, large language models (LLMs) have exhibited impressive results in diverse biomedical NLP tasks due to their unprecedentedly rich prior knowledge and strong zero-shot prediction abilities. However, LLMs suffer from issues including high costs, limited context length, and unreliable predictions. In this research, we propose PromptLink, a novel biomedical concept linking framework that leverages LLMs. It first employs a biomedical-specialized pre-trained language model to generate candidate concepts that can fit in the LLM context windows. Then it utilizes an LLM to link concepts through two-stage prompts, where the first-stage prompt aims to elicit the biomedical prior knowledge from the LLM for the concept linking task and the second-stage prompt enforces the LLM to reflect on its own predictions to further enhance their reliability. Empirical results on the concept linking task between two EHR datasets and an external biomedical KG demonstrate the effectiveness of PromptLink. Furthermore, PromptLink is a generic framework without reliance on additional prior knowledge, context, or training data, making it well-suited for concept linking across various types of data sources. The source code is available at https://github.com/constantjxyz/PromptLink.
Paper Structure (10 sections, 3 equations, 3 figures, 3 tables)

This paper contains 10 sections, 3 equations, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Biomedical Concept Linking
  3. Problem Definition
  4. PromptLink
  5. Experiments & Discussions
  6. Implementation Details
  7. Concept Linking Experiment Results
  8. Ablation Studies
  9. Case Studies
  10. Conclusion

Figures (3)

  • Figure 1: A toy example of biomedical concept linking. Left: concepts in the EHR. Right: concepts in the biomedical KG.
  • Figure 2: Overview of our proposed PromptLink framework.
  • Figure 3: Accuracy on MIID-NIL: Traditional ML-based methods outputting matching scores have varying NIL prediction performance based on the selected threshold, while PromptLink does not need a threshold yet consistently performs better.

Theorems & Definitions (3)

  • Definition 1: EHR
  • Definition 2: Biomedical KG
  • Definition 3: Biomedical Concept Linking