Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Overview of TREC 2025 Biomedical Generative Retrieval (BioGen) Track

Deepak Gupta, Dina Demner-Fushman, William Hersh, Steven Bedrick, Kirk Roberts

Abstract

Recent advances in large language models (LLMs) have made significant progress across multiple biomedical tasks, including biomedical question answering, lay-language summarization of the biomedical literature, and clinical note summarization. These models have demonstrated strong capabilities in processing and synthesizing complex biomedical information and in generating fluent, human-like responses. Despite these advancements, hallucinations or confabulations remain key challenges when using LLMs in biomedical and other high-stakes domains. Inaccuracies may be particularly harmful in high-risk situations, such as medical question answering, making clinical decisions, or appraising biomedical research. Studies on the evaluation of the LLMs' abilities to ground generated statements in verifiable sources have shown that models perform significantly

Overview of TREC 2025 Biomedical Generative Retrieval (BioGen) Track

Abstract

Recent advances in large language models (LLMs) have made significant progress across multiple biomedical tasks, including biomedical question answering, lay-language summarization of the biomedical literature, and clinical note summarization. These models have demonstrated strong capabilities in processing and synthesizing complex biomedical information and in generating fluent, human-like responses. Despite these advancements, hallucinations or confabulations remain key challenges when using LLMs in biomedical and other high-stakes domains. Inaccuracies may be particularly harmful in high-risk situations, such as medical question answering, making clinical decisions, or appraising biomedical research. Studies on the evaluation of the LLMs' abilities to ground generated statements in verifiable sources have shown that models perform significantly
Paper Structure (20 sections, 4 figures, 12 tables)

This paper contains 20 sections, 4 figures, 12 tables.

Table of Contents

  1. Overview
  2. Tasks
  3. Topics
  4. Data
  5. Participating Teams and Submissions
  6. Baseline Approaches
  7. Assessment
  8. Task A (Grounding Answer)
  9. Expert Evaluation
  10. Automatic Evaluation
  11. Task B (Reference Attribution)
  12. Expert Evaluation
  13. Answer quality
  14. Citation Quality
  15. Document relevancy
  16. ...and 5 more sections

Figures (4)

  • Figure 1: Sample reference answer for one of the topics of Task B of the BioGen track
  • Figure 3: (Task A) Comparison of the submitted runs using the expert evaluation scheme in terms of Precision (Strict and Relaxed) for the Supports class.
  • Figure 4: (Task A) Comparison of the submitted runs using the expert evaluation scheme in terms of SoftRecall (Strict and Relaxed) for the Supports class.
  • Figure 5: (Task A) Comparison of the submitted runs using the expert evaluation scheme in terms of Precision and SoftRecall for the Contradicts class. Please note that for the Contradicts class, strict and relaxed results remain the same.