Overview of the TREC 2025 Retrieval Augmented Generation (RAG) Track

TL;DR

This year's TREC 2025 RAG Track introduces long, multi-sentence narrative queries to better reflect the deep search task with the growing demand for reasoning-driven responses.

Abstract

The second edition of the TREC Retrieval Augmented Generation (RAG) Track advances research on systems that integrate retrieval and generation to address complex, real-world information needs. Building on the foundation of the inaugural 2024 track, this year's challenge introduces long, multi-sentence narrative queries to better reflect the deep search task with the growing demand for reasoning-driven responses. Participants are tasked with designing pipelines that combine retrieval and generation while ensuring transparency and factual grounding. The track leverages the MS MARCO V2.1 corpus and employs a multi-layered evaluation framework encompassing relevance assessment, response completeness, attribution verification, and agreement analysis. By emphasizing multi-faceted narratives and attribution-rich answers from over 150 submissions this year, the TREC 2025 RAG Track aims to foster innovation in creating trustworthy, context-aware systems for retrieval augmented generation.

Figures (8)

• Figure 1: Overview of the narrative generation process in the TREC 2025 RAG Track.
• Figure 2: Prompt used in the first step of the sub-narrative generation process with GPT-4.1 nano to generate an exhaustive list of sub-queries using the given narrative and query cluster as input.
• Figure 3: Prompt used in the second step of refining the sub-narratives with GPT-4.1 to generate an exhaustive list of sub-queries using the given narrative and the sub-queries generated in the previous step as input.
• Figure 4: Example of narrative and sub-narrative pair used in TREC 2025 RAG.
• Figure 5: Prompt for assigning relevance labels, where the inputs are the sub-narrative list, narrative and passage pair.