Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Multi-Disciplinary Dataset Discovery from Citation-Verified Literature Contexts

Zhiyin Tan, Changxu Duan

TL;DR

The paper tackles the challenge of discovering datasets by relying on metadata, which often fails to capture the actual research needs. It introduces a literature-driven framework that mines citation contexts to identify datasets used or referenced in prior work, treating papers as semantic bridges between research questions and data resources. A three-stage pipeline—scalable citation-context retrieval, LLM-based dataset mention extraction with citation-aware quality, and deterministic entity resolution—enables cross-domain discovery with provenance preservation. Automated CS benchmarks and cross-disciplinary expert evaluations show substantially higher recall and demonstrated utility and novelty of surfaced datasets compared to metadata-based baselines, with code and data released for reproducibility. This approach advances practical, context-aware dataset discovery and points toward broader scholarly knowledge graphs that link data, methods, and tools.

Abstract

Identifying suitable datasets for a research question remains challenging because existing dataset search engines rely heavily on metadata quality and keyword overlap, which often fail to capture the semantic intent of scientific investigation. We introduce a literature-driven framework that discovers datasets from citation contexts in scientific papers, enabling retrieval grounded in actual research use rather than metadata availability. Our approach combines large-scale citation-context extraction, schema-guided dataset recognition with Large Language Models, and provenance-preserving entity resolution. We evaluate the system on eight survey-derived computer science queries and find that it achieves substantially higher recall than Google Dataset Search and DataCite Commons, with normalized recall ranging from an average of 47.47% to a highest value of 81.82%. Beyond recovering gold-standard datasets, the method also surfaces additional datasets not documented in the surveys. Expert assessments across five top-level Fields of Science indicate that a substantial portion of the additional datasets are considered high utility, and some are regarded as novel for the specific topics chosen by the experts. These findings establish citation-context mining as an effective and generalizable paradigm for dataset discovery, particularly in settings where datasets lack sufficient or reliable metadata. To support reproducibility and future extensions, we release our code, evaluation datasets, and results on GitHub (https://github.com/Fireblossom/citation-context-dataset-discovery).

Multi-Disciplinary Dataset Discovery from Citation-Verified Literature Contexts

TL;DR

The paper tackles the challenge of discovering datasets by relying on metadata, which often fails to capture the actual research needs. It introduces a literature-driven framework that mines citation contexts to identify datasets used or referenced in prior work, treating papers as semantic bridges between research questions and data resources. A three-stage pipeline—scalable citation-context retrieval, LLM-based dataset mention extraction with citation-aware quality, and deterministic entity resolution—enables cross-domain discovery with provenance preservation. Automated CS benchmarks and cross-disciplinary expert evaluations show substantially higher recall and demonstrated utility and novelty of surfaced datasets compared to metadata-based baselines, with code and data released for reproducibility. This approach advances practical, context-aware dataset discovery and points toward broader scholarly knowledge graphs that link data, methods, and tools.

Abstract

Identifying suitable datasets for a research question remains challenging because existing dataset search engines rely heavily on metadata quality and keyword overlap, which often fail to capture the semantic intent of scientific investigation. We introduce a literature-driven framework that discovers datasets from citation contexts in scientific papers, enabling retrieval grounded in actual research use rather than metadata availability. Our approach combines large-scale citation-context extraction, schema-guided dataset recognition with Large Language Models, and provenance-preserving entity resolution. We evaluate the system on eight survey-derived computer science queries and find that it achieves substantially higher recall than Google Dataset Search and DataCite Commons, with normalized recall ranging from an average of 47.47% to a highest value of 81.82%. Beyond recovering gold-standard datasets, the method also surfaces additional datasets not documented in the surveys. Expert assessments across five top-level Fields of Science indicate that a substantial portion of the additional datasets are considered high utility, and some are regarded as novel for the specific topics chosen by the experts. These findings establish citation-context mining as an effective and generalizable paradigm for dataset discovery, particularly in settings where datasets lack sufficient or reliable metadata. To support reproducibility and future extensions, we release our code, evaluation datasets, and results on GitHub (https://github.com/Fireblossom/citation-context-dataset-discovery).
Paper Structure (55 sections, 3 equations, 3 figures, 4 tables, 2 algorithms)

This paper contains 55 sections, 3 equations, 3 figures, 4 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Dataset Search Engines and Repositories
  4. Literature-Based Information Extraction
  5. Citation Context Analysis
  6. Methodology
  7. Problem Formulation
  8. Three-Stage Pipeline
  9. Stage 1: Scalable citation-context retrieval
  10. Stage 2: LLM-based dataset mention extraction with citation-aware quality
  11. Stage 3: Dataset entity resolution and integration
  12. Evaluation Framework and Quality Metrics
  13. Extraction quality
  14. Evidence quality
  15. Practical accessibility and efficiency
  16. ...and 40 more sections

Figures (3)

  • Figure 1: Bridging research questions to datasets via citation contexts.
  • Figure 2: The proposed pipeline maps a research question to relevant datasets via citation context analysis and metadata linking.
  • Figure 3: Expert evaluation ratings across six cross-disciplinary research questions. Each radar chart visualizes the mean user ratings for a single query, comparing our system against the baselines across six dimensions of quality. The detailed numerical results are available in Table \ref{['tab:expert_all_rqs_single']}.