Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Ontology-based Semantic Similarity Measures for Clustering Medical Concepts in Drug Safety

Jeffery L Painter, François Haguinet, Gregory E Powell, Andrew Bate

TL;DR

This work tackles improving pharmacovigilance by clustering MedDRA adverse-event terms using ontology-based semantic similarity measures. It benchmarks six SSMs over a MedDRA+SNOMED-CT ontology network via UMLS and delivers a scalable high-throughput framework with cross-language interfaces. The main finding is that intrinsic information content-based measures—particularly INTRINSIC_LIN, INTRINSIC_LCH, and SOKAL—achieve the best clustering accuracy (F1 around 0.403–0.404) and outperform path-based methods, validated against expert reviews and SMQs. The approach offers a practical, ontology-grounded enhancement to automated safety signal detection, potentially reducing manual review workflows in pharmacovigilance.

Abstract

Semantic similarity measures (SSMs) are widely used in biomedical research but remain underutilized in pharmacovigilance. This study evaluates six ontology-based SSMs for clustering MedDRA Preferred Terms (PTs) in drug safety data. Using the Unified Medical Language System (UMLS), we assess each method's ability to group PTs around medically meaningful centroids. A high-throughput framework was developed with a Java API and Python and R interfaces support large-scale similarity computations. Results show that while path-based methods perform moderately with F1 scores of 0.36 for WUPALMER and 0.28 for LCH, intrinsic information content (IC)-based measures, especially INTRINSIC-LIN and SOKAL, consistently yield better clustering accuracy (F1 score of 0.403). Validated against expert review and standard MedDRA queries (SMQs), our findings highlight the promise of IC-based SSMs in enhancing pharmacovigilance workflows by improving early signal detection and reducing manual review.

Ontology-based Semantic Similarity Measures for Clustering Medical Concepts in Drug Safety

TL;DR

This work tackles improving pharmacovigilance by clustering MedDRA adverse-event terms using ontology-based semantic similarity measures. It benchmarks six SSMs over a MedDRA+SNOMED-CT ontology network via UMLS and delivers a scalable high-throughput framework with cross-language interfaces. The main finding is that intrinsic information content-based measures—particularly INTRINSIC_LIN, INTRINSIC_LCH, and SOKAL—achieve the best clustering accuracy (F1 around 0.403–0.404) and outperform path-based methods, validated against expert reviews and SMQs. The approach offers a practical, ontology-grounded enhancement to automated safety signal detection, potentially reducing manual review workflows in pharmacovigilance.

Abstract

Semantic similarity measures (SSMs) are widely used in biomedical research but remain underutilized in pharmacovigilance. This study evaluates six ontology-based SSMs for clustering MedDRA Preferred Terms (PTs) in drug safety data. Using the Unified Medical Language System (UMLS), we assess each method's ability to group PTs around medically meaningful centroids. A high-throughput framework was developed with a Java API and Python and R interfaces support large-scale similarity computations. Results show that while path-based methods perform moderately with F1 scores of 0.36 for WUPALMER and 0.28 for LCH, intrinsic information content (IC)-based measures, especially INTRINSIC-LIN and SOKAL, consistently yield better clustering accuracy (F1 score of 0.403). Validated against expert review and standard MedDRA queries (SMQs), our findings highlight the promise of IC-based SSMs in enhancing pharmacovigilance workflows by improving early signal detection and reducing manual review.

Paper Structure

This paper contains 20 sections, 6 equations, 6 figures, 1 table.

Table of Contents

  1. Introduction
  2. Methods
  3. Semantic Similarity Measures Evaluated
  4. Leacock and Chodorow (LCH)
  5. Wu and Palmer (WUPALMER)
  6. Resnik (INTRINSIC_RESNIK)
  7. Lin (INTRINSIC_LIN)
  8. Intrinsic Leacock and Chodorow (INTRINSIC_LCH)
  9. Sokal (SOKAL)
  10. Evaluation of SSMs
  11. Metric Selection
  12. Medical Review and Reference Standards
  13. Constructing the Review Datasets
  14. Performance Metrics and Review Process
  15. Review Process and Adjudication
  16. ...and 5 more sections

Figures (6)

  • Figure 1: Feasibility analysis of SSMs and ontological views. MDR: MedDRA; MSH: MeSH; SCT: SNOMED-CT; UMLS: Unified Medical Language System
  • Figure 2: Process for constructing evaluation dataset
  • Figure 3: Cohen’s Kappa comparing medical review to SMQ and HLGT groupings
  • Figure 4: SSM Distributions
  • Figure 5: ROC Curve Analyses
  • ...and 1 more figures