Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SCP-116K: A High-Quality Problem-Solution Dataset and a Generalized Pipeline for Automated Extraction in the Higher Education Science Domain

Dakuan Lu, Xiaoyu Tan, Rui Xu, Tianchu Yao, Chao Qu, Wei Chu, Yinghui Xu, Yuan Qi

TL;DR

The paper introduces SCP-116K, a large-scale dataset of 116,756 high-quality science problem-solution pairs designed for higher education contexts. It presents a generalizable six-stage automated pipeline to extract, normalize, segment, and pair problems with solutions from diverse sources, including image-to-LaTeX handling and rigorous quality filtering. Through baseline evaluations and knowledge distillation experiments, the work demonstrates substantial improvements on challenging scientific reasoning benchmarks and shows that SCP-116K can bridge the gap toward larger models. By open-sourcing both the dataset and the extraction pipeline, the authors aim to catalyze research in scientific reasoning and enable scalable development of domain-specific LLMs for STEM disciplines.

Abstract

Recent breakthroughs in large language models (LLMs) exemplified by the impressive mathematical and scientific reasoning capabilities of the o1 model have spotlighted the critical importance of high-quality training data in advancing LLM performance across STEM disciplines. While the mathematics community has benefited from a growing body of curated datasets, the scientific domain at the higher education level has long suffered from a scarcity of comparable resources. To address this gap, we present SCP-116K, a new large-scale dataset of 116,756 high-quality problem-solution pairs, automatically extracted from heterogeneous sources using a streamlined and highly generalizable pipeline. Our approach involves stringent filtering to ensure the scientific rigor and educational level of the extracted materials, while maintaining adaptability for future expansions or domain transfers. By openly releasing both the dataset and the extraction pipeline, we seek to foster research on scientific reasoning, enable comprehensive performance evaluations of new LLMs, and lower the barrier to replicating the successes of advanced models like o1 in the broader science community. We believe SCP-116K will serve as a critical resource, catalyzing progress in high-level scientific reasoning tasks and promoting further innovations in LLM development. The dataset and code are publicly available at https://github.com/AQA6666/SCP-116K-open.

SCP-116K: A High-Quality Problem-Solution Dataset and a Generalized Pipeline for Automated Extraction in the Higher Education Science Domain

TL;DR

The paper introduces SCP-116K, a large-scale dataset of 116,756 high-quality science problem-solution pairs designed for higher education contexts. It presents a generalizable six-stage automated pipeline to extract, normalize, segment, and pair problems with solutions from diverse sources, including image-to-LaTeX handling and rigorous quality filtering. Through baseline evaluations and knowledge distillation experiments, the work demonstrates substantial improvements on challenging scientific reasoning benchmarks and shows that SCP-116K can bridge the gap toward larger models. By open-sourcing both the dataset and the extraction pipeline, the authors aim to catalyze research in scientific reasoning and enable scalable development of domain-specific LLMs for STEM disciplines.

Abstract

Recent breakthroughs in large language models (LLMs) exemplified by the impressive mathematical and scientific reasoning capabilities of the o1 model have spotlighted the critical importance of high-quality training data in advancing LLM performance across STEM disciplines. While the mathematics community has benefited from a growing body of curated datasets, the scientific domain at the higher education level has long suffered from a scarcity of comparable resources. To address this gap, we present SCP-116K, a new large-scale dataset of 116,756 high-quality problem-solution pairs, automatically extracted from heterogeneous sources using a streamlined and highly generalizable pipeline. Our approach involves stringent filtering to ensure the scientific rigor and educational level of the extracted materials, while maintaining adaptability for future expansions or domain transfers. By openly releasing both the dataset and the extraction pipeline, we seek to foster research on scientific reasoning, enable comprehensive performance evaluations of new LLMs, and lower the barrier to replicating the successes of advanced models like o1 in the broader science community. We believe SCP-116K will serve as a critical resource, catalyzing progress in high-level scientific reasoning tasks and promoting further innovations in LLM development. The dataset and code are publicly available at https://github.com/AQA6666/SCP-116K-open.
Paper Structure (25 sections, 1 figure, 1 table)

This paper contains 25 sections, 1 figure, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. Scientific Question-Answering Datasets
  4. Benchmarking Standards
  5. Advanced Scientific Reasoning Models
  6. Methodology
  7. Document Retrieval and Filtering
  8. Unified Preprocessing
  9. Content Segmentation
  10. Structured Extraction
  11. Quality Filtering
  12. Problem-Solution Matching
  13. Model Response Collection
  14. Experiments
  15. Baseline Performance on SCP-116K
  16. ...and 10 more sections

Figures (1)

  • Figure 1: Overview of our automated pipeline for scientific problem-solution pair extraction. The pipeline consists of six main stages: (1) document retrieval and filtering, (2) unified preprocessing, (3) segmentation, (4) extraction, (5) quality filtering, and (6) problem-solution matching. Each stage is designed to maintain high data quality while ensuring scalability and generalizability across different scientific domains.