Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

BIOME-Bench: A Benchmark for Biomolecular Interaction Inference and Multi-Omics Pathway Mechanism Elucidation from Scientific Literature

Sibo Wei, Peng Chen, Lifeng Dong, Yin Luo, Lei Wang, Peng Zhang, Wenpeng Lu, Jianbin Guo, Hongjun Yang, Dajun Zeng

TL;DR

BIOME-Bench presents a literature-grounded benchmark for end-to-end multi-omics pathway mechanism elucidation, addressing limitations of traditional PE approaches and ad hoc case studies. It introduces a rigorous four-stage data-construction workflow to generate instance-level supervision and defines two evaluation tasks: biomolecular interaction inference and multi-omics pathway mechanism elucidation. Across a broad set of contemporary LLMs, BIOME-Bench reveals substantial gaps in fine-grained interaction discrimination and robust, state-aware mechanistic explanation generation, despite reasonable factual grounding and coherence. The framework enables targeted diagnostics of model behavior, supports human expert validation, and provides a foundation for reproducible progress in mechanistic interpretation from multi-omics literature.

Abstract

Multi-omics studies often rely on pathway enrichment to interpret heterogeneous molecular changes, but pathway enrichment (PE)-based workflows inherit structural limitations of pathway resources, including curation lag, functional redundancy, and limited sensitivity to molecular states and interventions. Although recent work has explored using large language models (LLMs) to improve PE-based interpretation, the lack of a standardized benchmark for end-to-end multi-omics pathway mechanism elucidation has largely confined evaluation to small, manually curated datasets or ad hoc case studies, hindering reproducible progress. To address this issue, we introduce BIOME-Bench, constructed via a rigorous four-stage workflow, to evaluate two core capabilities of LLMs in multi-omics analysis: Biomolecular Interaction Inference and end-to-end Multi-Omics Pathway Mechanism Elucidation. We develop evaluation protocols for both tasks and conduct comprehensive experiments across multiple strong contemporary models. Experimental results demonstrate that existing models still exhibit substantial deficiencies in multi-omics analysis, struggling to reliably distinguish fine-grained biomolecular relation types and to generate faithful, robust pathway-level mechanistic explanations.

BIOME-Bench: A Benchmark for Biomolecular Interaction Inference and Multi-Omics Pathway Mechanism Elucidation from Scientific Literature

TL;DR

BIOME-Bench presents a literature-grounded benchmark for end-to-end multi-omics pathway mechanism elucidation, addressing limitations of traditional PE approaches and ad hoc case studies. It introduces a rigorous four-stage data-construction workflow to generate instance-level supervision and defines two evaluation tasks: biomolecular interaction inference and multi-omics pathway mechanism elucidation. Across a broad set of contemporary LLMs, BIOME-Bench reveals substantial gaps in fine-grained interaction discrimination and robust, state-aware mechanistic explanation generation, despite reasonable factual grounding and coherence. The framework enables targeted diagnostics of model behavior, supports human expert validation, and provides a foundation for reproducible progress in mechanistic interpretation from multi-omics literature.

Abstract

Multi-omics studies often rely on pathway enrichment to interpret heterogeneous molecular changes, but pathway enrichment (PE)-based workflows inherit structural limitations of pathway resources, including curation lag, functional redundancy, and limited sensitivity to molecular states and interventions. Although recent work has explored using large language models (LLMs) to improve PE-based interpretation, the lack of a standardized benchmark for end-to-end multi-omics pathway mechanism elucidation has largely confined evaluation to small, manually curated datasets or ad hoc case studies, hindering reproducible progress. To address this issue, we introduce BIOME-Bench, constructed via a rigorous four-stage workflow, to evaluate two core capabilities of LLMs in multi-omics analysis: Biomolecular Interaction Inference and end-to-end Multi-Omics Pathway Mechanism Elucidation. We develop evaluation protocols for both tasks and conduct comprehensive experiments across multiple strong contemporary models. Experimental results demonstrate that existing models still exhibit substantial deficiencies in multi-omics analysis, struggling to reliably distinguish fine-grained biomolecular relation types and to generate faithful, robust pathway-level mechanistic explanations.
Paper Structure (34 sections, 11 equations, 4 figures, 2 tables)

This paper contains 34 sections, 11 equations, 4 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Pathway Knowledge Bases
  4. Literature-Grounded Benchmarks for Pathway Reasoning and Relation Extraction
  5. LLM-Based Systems for Multi-Omics and Pathway Interpretation
  6. Methodology
  7. Phase I: Literature Retrieval and Relevance Filtering
  8. MeSH-guided Literature Retrieval.
  9. LLM-based Semantic and Mechanistic Relevance Scoring.
  10. Phase II: Information Extraction and Entity Standardization
  11. LLM-based Mechanistic Extraction
  12. Entity Normalization and Ontology Mapping
  13. Phase III: Knowledge Structuring and Validation
  14. Interaction Quadruple Extraction
  15. Biological State Annotation
  16. ...and 19 more sections

Figures (4)

  • Figure 1: Overview of a pathway enrichment-based multi-omics mechanism elucidation workflow. Multi-omics experiments (e.g., metabolomics, proteomics, and scRNA-seq) are followed by differential analysis with significance filtering to identify perturbed entities, which are then mapped to biological pathways via enrichment analysis to support downstream mechanistic interpretation.
  • Figure 2: Workflow for constructing BIOME-Bench: (I) MeSH-guided PubMed retrieval with LLM relevance filtering; (II) LLM-based entity extraction and standardization; (III) state-aware knowledge graph construction with human sampling verification; and (IV) benchmark formulation with two tasks—biomolecular interaction inference and multi-omics pathway mechanism elucidation.
  • Figure 3: Sensitivity of Qwen3-32B judge to semantic perturbations. Scores are reported for rewrite and perturb . Drop% denotes the relative score decrease from rewrite to perturb.
  • Figure 4: Error confusion matrix for Biomolecular Interaction Inference. Rows are gold relation types and columns are predicted types. Color encodes the count of misclassified gold$\rightarrow$predicted relations.