Grounding Synthetic Data Evaluations of Language Models in Unsupervised Document Corpora

TL;DR

This work tackles the scalability gap in evaluating language models for domain-specific knowledge by proposing an automated pipeline that generates fact-based benchmarks grounded in authoritative documents. By chunking documents, extracting topics, and producing both multiple-choice and open-ended questions with grounded explanations, the approach enables rapid, domain-relevant evaluations that can be aligned with human benchmarks. Across six NLP QA datasets, synthetic benchmarks show strong alignment with human data, achieving a Spearman rank correlation near $0.97$ and a Pearson accuracy correlation around $0.75$, with case studies highlighting robust OE performance for certain models. The methodology offers a practical path to targeted LM evaluation in professional domains, while also revealing limitations such as tendency for longer questions to inflate scores and the need for richer content sources and chunking strategies.

Abstract

Language Models (LMs) continue to advance, improving response quality and coherence. Given Internet-scale training datasets, LMs have likely encountered much of what users may ask them to generate in some form during their training. A plethora of evaluation benchmarks have been constructed to assess model quality, response appropriateness, and reasoning capabilities. However, the human effort required for benchmark construction is rapidly being outpaced by the size and scope of the models under evaluation. Having humans build a benchmark for every possible domain of interest is impractical. Therefore, we propose a methodology for automating the construction of fact-based synthetic data model evaluations grounded in document populations. This work leverages the same LMs to evaluate domain-specific knowledge automatically, using only grounding documents (e.g., a textbook) as input. This synthetic data benchmarking approach corresponds well with human curated questions producing a Spearman ranking correlation of 0.97 and a benchmark evaluation Pearson accuracy correlation of 0.75. This novel approach supports generating both multiple choice and open-ended synthetic data questions to gain diagnostic insight of LM capability. We apply this methodology to evaluate model performance on two recent arXiv preprints, discovering a surprisingly strong performance from Gemma-3 models on open-ended questions. Code is available at https://github.com/mmajurski/grounded-synth-lm-benchmark

Figures (14)

• Figure 1: Synthetic evaluation pipeline: (1) users curate grounding documents, (2) LMs generate domain-specific questions, (3) model responses are evaluated.
• Figure 2: Comparison of average LM performance across six datasets with the multiple choice question synthetic data benchmarks created by the ensemble of bolded model in \ref{['tab:models']}.
• Figure 3: Synthetic data generated question correctness and explanation validity on a scale of 1 to 10.
• Figure 4: Generated question clarity and groundedness compared against the original questions.
• Figure 5: Impact of question length on evaluation accuracy over-performance.