ScrapeGraphAI-100k: A Large-Scale Dataset for LLM-Based Web Information Extraction

TL;DR

This work introduces ScrapeGraphAI-100k, a large-scale dataset comprising real-world LLM extraction events, collected via opt-in ScrapeGraphAI telemetry during Q2 and Q3 of 2025, and characterize the datasets structural diversity and its failure modes as schema complexity increases.

Abstract

The use of large language models for web information extraction is becoming increasingly fundamental to modern web information retrieval pipelines. However, existing datasets tend to be small, synthetic or text-only, failing to capture the structural context of the web. We introduce ScrapeGraphAI-100k, a large-scale dataset comprising real-world LLM extraction events, collected via opt-in ScrapeGraphAI telemetry during Q2 and Q3 of 2025. Starting from 9M events, we deduplicate and balance by schema to produce 93,695 examples spanning diverse domains and languages. Each instance includes Markdown content, a prompt, a JSON schema, the LLM response, and complexity/validation metadata. We characterize the datasets structural diversity and its failure modes as schema complexity increases. We also provide a fine-tuning experiment showing that a small language model (1.7B) trained on a subset narrows the gap to larger baselines (30B), underscoring the datasets utility for efficient extraction. ScrapeGraphAI-100k enables fine-tuning small models, benchmarking structured extraction, and studying schema induction for web IR indexing, and is publicly available on HuggingFace.

Figures (6)

• Figure 1: Schema complexity distributions in ScrapeGraphAI-100k (depth, key count, elements, cyclomatic complexity, composite score) across 93,695 extraction events, highlighting a dense core and a long tail of complex schemas.
• Figure 2: Schema validation rate versus complexity (depth, key count, composite score); the dashed line marks the corpus mean and shows declining validity at higher complexity.
• Figure 3: Response size distribution for 93,695 extraction outputs (log-scale y-axis), with most responses under 100 KB and a long tail approaching 1 MB.
• Figure 4: Overall BLEU vs. model size on the evaluation set. Circles are Qwen3 baselines (1.7B/4B/30B) and the star is our fine-tuned 1.7B model, which gains +25.7% over the 1.7B baseline and comes within 3.0% of the 30B model.
• Figure 5: Correlation matrix of dataset metrics, showing strong correlations among complexity measures and a negative relationship between complexity and validation.