Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

OpenDDI: A Comprehensive Benchmark for DDI Prediction

Xinmo Jin, Bowen Fan, Xunkai Li, Henan Sun, YuXin Zeng, Zekai Chen, Yuxuan Sun, Jia Li, Qiangqiang Dai, Hongchao Qin, Rong-Hua Li, Guoren Wang

TL;DR

OpenDDI tackles data quality and evaluation standardization in DDI prediction by introducing a comprehensive benchmark that unifies 6 existing datasets and 3 new large-scale, LLM-augmented datasets, along with a five-modality multimodal drug representation. It evaluates 20 state-of-the-art baselines across 3 downstream tasks (binary, multiclass, multilabel) using standardized pipelines across data quality, effectiveness, generalization, robustness, and efficiency, enabling fair, reproducible comparisons. The study yields 11 actionable insights, highlighting the strengths of graph-based and multimodal approaches, the challenges of generalization to unseen drugs, and the trade-offs in efficiency and robustness. By providing open-source data and code, OpenDDI aims to accelerate reliable DDI prediction research and inform safer, more scalable pharmacovigilance and drug discovery efforts.

Abstract

Drug-Drug Interactions (DDIs) significantly influence therapeutic efficacy and patient safety. As experimental discovery is resource-intensive and time-consuming, efficient computational methodologies have become essential. The predominant paradigm formulates DDI prediction as a drug graph-based link prediction task. However, further progress is hindered by two fundamental challenges: (1) lack of high-quality data: most studies rely on small-scale DDI datasets and single-modal drug representations; (2) lack of standardized evaluation: inconsistent scenarios, varied metrics, and diverse baselines. To address the above issues, we propose OpenDDI, a comprehensive benchmark for DDI prediction. Specifically, (1) from the data perspective, OpenDDI unifies 6 widely used DDI datasets and 2 existing forms of drug representation, while additionally contributing 3 new large-scale LLM-augmented datasets and a new multimodal drug representation covering 5 modalities. (2) From the evaluation perspective, OpenDDI unifies 20 SOTA model baselines across 3 downstream tasks, with standardized protocols for data quality, effectiveness, generalization, robustness, and efficiency. Based on OpenDDI, we conduct a comprehensive evaluation and derive 10 valuable insights for DDI prediction while exposing current limitations to provide critical guidance for this rapidly evolving field. Our code is available at https://github.com/xiaoriwuguang/OpenDDI

OpenDDI: A Comprehensive Benchmark for DDI Prediction

TL;DR

OpenDDI tackles data quality and evaluation standardization in DDI prediction by introducing a comprehensive benchmark that unifies 6 existing datasets and 3 new large-scale, LLM-augmented datasets, along with a five-modality multimodal drug representation. It evaluates 20 state-of-the-art baselines across 3 downstream tasks (binary, multiclass, multilabel) using standardized pipelines across data quality, effectiveness, generalization, robustness, and efficiency, enabling fair, reproducible comparisons. The study yields 11 actionable insights, highlighting the strengths of graph-based and multimodal approaches, the challenges of generalization to unseen drugs, and the trade-offs in efficiency and robustness. By providing open-source data and code, OpenDDI aims to accelerate reliable DDI prediction research and inform safer, more scalable pharmacovigilance and drug discovery efforts.

Abstract

Drug-Drug Interactions (DDIs) significantly influence therapeutic efficacy and patient safety. As experimental discovery is resource-intensive and time-consuming, efficient computational methodologies have become essential. The predominant paradigm formulates DDI prediction as a drug graph-based link prediction task. However, further progress is hindered by two fundamental challenges: (1) lack of high-quality data: most studies rely on small-scale DDI datasets and single-modal drug representations; (2) lack of standardized evaluation: inconsistent scenarios, varied metrics, and diverse baselines. To address the above issues, we propose OpenDDI, a comprehensive benchmark for DDI prediction. Specifically, (1) from the data perspective, OpenDDI unifies 6 widely used DDI datasets and 2 existing forms of drug representation, while additionally contributing 3 new large-scale LLM-augmented datasets and a new multimodal drug representation covering 5 modalities. (2) From the evaluation perspective, OpenDDI unifies 20 SOTA model baselines across 3 downstream tasks, with standardized protocols for data quality, effectiveness, generalization, robustness, and efficiency. Based on OpenDDI, we conduct a comprehensive evaluation and derive 10 valuable insights for DDI prediction while exposing current limitations to provide critical guidance for this rapidly evolving field. Our code is available at https://github.com/xiaoriwuguang/OpenDDI
Paper Structure (45 sections, 7 equations, 6 figures, 6 tables, 3 algorithms)

This paper contains 45 sections, 7 equations, 6 figures, 6 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Definitions and Background
  3. Benchmark Design
  4. Algorithm Framework
  5. Dataset Curation
  6. Evaluation Procedure
  7. Experiments and Analyses
  8. Datasets Comparison
  9. Datasets and Drug Representations Evaluation
  10. Multimodal Research
  11. Model Performance Comparison
  12. Generalization Analyses
  13. Efficiency Analyses
  14. Robustness Analyses
  15. Conclusion and Future Directions
  16. ...and 30 more sections

Figures (6)

  • Figure 1: Comparison of previous and our newly-built datasets.
  • Figure 2: An overview of the OpenDDI Framework.
  • Figure 3: F1-score(%) comparison for DDI multiclass-prediction task after dissolve a modality.
  • Figure 4: F1-score(%) comparison for DDI multiclass-prediction task with SMILES and another modality.
  • Figure 5: Efficiency Performance on Various Datasets.
  • ...and 1 more figures