BibAgent: An Agentic Framework for Traceable Miscitation Detection in Scientific Literature

TL;DR

BibAgent tackles the integrity of scientific citations by framing miscitation detection as a traceable, multi-stage reasoning task. It introduces a five-category Miscitation Taxonomy and MisciteBench to provide a robust, cross-disciplinary evaluation platform. BibAgent itself combines Accessible and Inaccessible Cited Source Verifiers (ACSV and ICSV) with adaptive retrieval, NLI, and a community-driven evidence committee to verify both open and paywalled sources, achieving strong performance gains and transparent reasoning trails. The work demonstrates scalable detection and explanation of miscitations, with practical implications for editor workflows and AI-assisted scholarly writing in the GenAI era.

Abstract

Citations are the bedrock of scientific authority, yet their integrity is compromised by widespread miscitations: ranging from nuanced distortions to fabricated references. Systematic citation verification is currently unfeasible; manual review cannot scale to modern publishing volumes, while existing automated tools are restricted by abstract-only analysis or small-scale, domain-specific datasets in part due to the "paywall barrier" of full-text access. We introduce BibAgent, a scalable, end-to-end agentic framework for automated citation verification. BibAgent integrates retrieval, reasoning, and adaptive evidence aggregation, applying distinct strategies for accessible and paywalled sources. For paywalled references, it leverages a novel Evidence Committee mechanism that infers citation validity via downstream citation consensus. To support systematic evaluation, we contribute a 5-category Miscitation Taxonomy and MisciteBench, a massive cross-disciplinary benchmark comprising 6,350 miscitation samples spanning 254 fields. Our results demonstrate that BibAgent outperforms state-of-the-art Large Language Model (LLM) baselines in citation verification accuracy and interpretability, providing scalable, transparent detection of citation misalignments across the scientific literature.

Figures (2)

• Figure 1: Overview of the Inaccessible Cited Source Verifier (ICSV) and its Evidence Committee mechanism. Given a citing context about a paywalled source $B$, ICSV (1) extracts an atomic claim that captures exactly what the citing paper attributes to $B$; (2) retrieves open-access downstream citers of $B$ and clusters their local citation contexts into aspect-specific groups; (3) distills each group into a canonical evidence statement, weighting it by a field-normalized influence score that combines venue and paper-level impact; and (4) aggregates the resulting entailment/contradiction/neutral votes into a reliability-aware consensus verdict, explicitly abstaining when community evidence is too sparse or internally inconsistent. This converts paywalled miscitation detection from an inaccessible-document problem into a traceable, community-consensus reasoning task.
• Figure 2: Evidence Committee behavior as a function of the number of distinct witness papers $n_{\text{voter}}$ supporting the dominant evidence statement $e^\star$ for a paywalled citation. Curves show non-abstention rate, conditional accuracy, and mean calibrated confidence (Appendix \ref{['app:icsv']}); shaded bands indicate variation across backbones. The sharp and stable transition around $n_{\text{voter}} = 6$ motivates the choice $K_{\min} = 6$ in ICSV's reliability-aware abstention rule.