From Knowledge to Inference: Scaling Laws of Specialized Reasoning on GlobalHealthAtlas

TL;DR

Public health reasoning in LLMs lags due to data scarcity and non-domain-specific evaluation. The authors propose GlobalHealthAtlas, a large-scale multilingual resource with $280,210$ instances across 15 domains and 17 languages, plus an evidence-centric data pipeline, a six-dimension domain-aligned evaluator, and a domain-tuned Public-Model via LoRA on $qwen$-series. Key contributions include a principled data construction paradigm, a rigorous evaluator with six orthogonal criteria, and extensive cross-domain, cross-lingual benchmarking demonstrating that domain-aligned fine-tuning markedly improves reasoning and robustness. The work offers a reproducible foundation for training and evaluating LLMs in safety-critical public health contexts, with insights into data scaling, transfer, and perturbation resilience. The scoring mechanism blends multiple facets of reasoning and factual grounding as $S = 0.45Q + 0.25A + 0.20R + 0.10C$, enabling nuanced assessment aligned with public health standards.

Abstract

Public health reasoning requires population level inference grounded in scientific evidence, expert consensus, and safety constraints. However, it remains underexplored as a structured machine learning problem with limited supervised signals and benchmarks. We introduce \textbf{GlobalHealthAtlas}, a large scale multilingual dataset of 280,210 instances spanning 15 public health domains and 17 languages, stratified into three difficulty levels from health literacy to epidemiological and policy reasoning. Instances are derived from openly available public health sources and labeled by language, domain, and difficulty to support supervised learning and slice based evaluation. We further propose large language model (LLM) assisted construction and quality control pipeline with retrieval, duplication, evidence grounding checks, and label validation to improve consistency at scale. Finally, we present a domain aligned evaluator distilled from high confidence judgments of diverse LLMs to assess outputs along six dimensions: Accuracy, Reasoning, Completeness, Consensus Alignment, Terminology Norms, and Insightfulness. Together, these contributions enable reproducible training and evaluation of LLMs for safety critical public health reasoning beyond conventional QA benchmarks.

Figures (3)

• Figure 1: GlobalHealthAtlas domain composition (left) prioritizes high-impact areas like infectious diseases and health policy, balanced by a diverse long-tail of complementary fields. The linguistic distribution (right) is anchored in English and extends across 17 languages to facilitate cross-lingual public health reasoning.
• Figure 2: The evidence-centric construction pipeline from raw data collection to multi-stage quality control of GlobalHealthAtlas.
• Figure 3: Performance variation across question difficulty levels under different fine-tuning regimes. The figure illustrates how model reasoning capability changes with increasing question difficulty from C (Popular Science / Public Awareness) to B (General Knowledge) and A (Academic / Professional), evaluated across multiple fine-tuning data fractions. Results are reported for qwen3-4b, qwen3-8b, and qwen3-14b, highlighting the interaction between difficulty stratification and fine-tuning scale.