Improving Scientific Document Retrieval with Academic Concept Index

TL;DR

The paper tackles the difficulty of applying general-domain retrievers to scientific corpora by introducing an academic concept index that captures core topics and phrases within documents. Building on this index, it proposes CCQGen, a concept coverage-based query generation method, and CCExpand, a training-free concept-focused context expansion method; both aim to improve conceptual alignment without extensive retraining. The concept index is enriched via a two-level extraction process (topics and phrases) and a concept extractor, enabling adaptive conditioning of LLMs and concept-grounded snippets for matching. Extensive experiments on CSFCube and DORIS-MAE demonstrate that these methods yield higher-quality, more diverse queries and concept-focused signals that boost retrieval performance across multiple backbones, while remaining efficient at inference. The results validate the value of modeling document concepts in scientific retrieval and suggest broader applicability to reranking and continual updating in real-world systems.

Abstract

Adapting general-domain retrievers to scientific domains is challenging due to the scarcity of large-scale domain-specific relevance annotations and the substantial mismatch in vocabulary and information needs. Recent approaches address these issues through two independent directions that leverage large language models (LLMs): (1) generating synthetic queries for fine-tuning, and (2) generating auxiliary contexts to support relevance matching. However, both directions overlook the diverse academic concepts embedded within scientific documents, often producing redundant or conceptually narrow queries and contexts. To address this limitation, we introduce an academic concept index, which extracts key concepts from papers and organizes them guided by an academic taxonomy. This structured index serves as a foundation for improving both directions. First, we enhance the synthetic query generation with concept coverage-based generation (CCQGen), which adaptively conditions LLMs on uncovered concepts to generate complementary queries with broader concept coverage. Second, we strengthen the context augmentation with concept-focused auxiliary contexts (CCExpand), which leverages a set of document snippets that serve as concise responses to the concept-aware CCQGen queries. Extensive experiments show that incorporating the academic concept index into both query generation and context augmentation leads to higher-quality queries, better conceptual alignment, and improved retrieval performance.

Figures (9)

• Figure 1: Overview of our framework. (a) We construct an academic concept index by extracting key concepts from each document. Leveraging this concept index, we explore two different directions for improving retrieval. (b) CCQGen uses the index to guide synthetic query generation by identifying uncovered concepts and producing complementary queries. (c) CCExpand generates concept-focused snippets that act as concise, concept-grounded views of the document, enabling fine-grained concept matching.
• Figure 2: Overview of the concept index construction. The illustration shows the indexing for a single document.
• Figure 3: The overview of Concept Coverage-based Query set Generation (CCQGen). Best viewed in color.
• Figure 4: An illustration of concept-focused snippet generation in CCExpand. Best viewed in color.
• Figure 5: An illustration of concept-focused relevance matching process. Best viewed in color.