D3: A Massive Dataset of Scholarly Metadata for Analyzing the State of Computer Science Research

TL;DR

The paper introduces the DBLP Discovery Dataset (D3), a large-scale, open dataset that augments the DBLP bibliographic corpus with full-text-derived metadata such as abstracts, author affiliations, and citations. By building a scalable extraction and alignment pipeline, the authors produce a ≈$6{,}392{,}734$ publication corpus enriched with rich metadata and a citation graph, enabling quantitative analyses of CS growth, collaboration, topic trends, and cross-field influence. Initial analyses show CS growth at roughly $15.12 ext{ extpercent}$ annually, increasing collaboration, and a decline in average citations for recent papers, while abstracts reveal emerging topics like transformer-based NLP models. D3 is positioned as a practical resource for scientometrics, language-model training, and venue/topic analyses, with future plans for REST APIs and interactive tools to broaden accessible analysis. The dataset, findings, and code are publicly available to support research into research trends and the structure of computer science scholarship.

Abstract

DBLP is the largest open-access repository of scientific articles on computer science and provides metadata associated with publications, authors, and venues. We retrieved more than 6 million publications from DBLP and extracted pertinent metadata (e.g., abstracts, author affiliations, citations) from the publication texts to create the DBLP Discovery Dataset (D3). D3 can be used to identify trends in research activity, productivity, focus, bias, accessibility, and impact of computer science research. We present an initial analysis focused on the volume of computer science research (e.g., number of papers, authors, research activity), trends in topics of interest, and citation patterns. Our findings show that computer science is a growing research field (approx. 15% annually), with an active and collaborative researcher community. While papers in recent years present more bibliographical entries in comparison to previous decades, the average number of citations has been declining. Investigating papers' abstracts reveals that recent topic trends are clearly reflected in D3. Finally, we list further applications of D3 and pose supplemental research questions. The D3 dataset, our findings, and source code are publicly available for research purposes.

Figures (8)

• Figure 1: The number of annual publications and authors in logarithmic scale between 1936 and 2021.
• Figure 2: A cubic approximation on the average number of authors per paper between 1936 and 2021.
• Figure 3: The number of authors and their published papers until December 2021.
• Figure 4: The relative amount of active researchers (colored and in %). An active researcher is defined by the minimum number of papers published (x-axis) in a number consecutive years (y-axis). For example, in the last 13 years, out of all researchers who published in that time, 45.95% published 2 or more papers.
• Figure 5: The most common terms in titles and abstracts between 1936 and 2021.