sui-1: Grounded and Verifiable Long-Form Summarization

TL;DR

Large language models frequently produce plausible yet unfaithful summaries lacking verifiability. The paper introduces sui-1, a 24B parameter model that outputs abstractive long-form summaries with inline citations, enabling users to trace claims to source sentences. The authors propose a synthetic data pipeline that combines chain-of-thought prompting with multi-stage verification across five languages, generating over 22k training examples. Empirical evaluation shows sui-1 outperforms all tested open-weight baselines and approaches a reference model, demonstrating that task-specific training can surpass scale alone for citation-grounded summarization. The work also releases model weights, a dataset, and an interactive demo to support reproducibility and practical deployment.

Abstract

Large language models frequently generate plausible but unfaithful summaries that users cannot verify against source text, a critical limitation in compliance-sensitive domains such as government and legal analysis. We present sui-1, a 24B parameter model that produces abstractive summaries with inline citations, enabling users to trace each claim to its source sentence. Our synthetic data pipeline combines chain-of-thought prompting with multi-stage verification, generating over 22,000 high-quality training examples across five languages from diverse sources including parliamentary documents, web text, and Wikipedia. Evaluation shows sui-1 significantly outperforms all tested open-weight baselines, including models with 3x more parameters. These results demonstrate that task-specific training substantially outperforms scale alone for citation-grounded summarization. Model weights and an interactive demo are publicly available.

Figures (7)

• Figure 1: Overall performance comparison. sui-1 (84.2%) significantly outperforms open-weight baselines and approaches the reference model (89.1%).
• Figure 2: Synthetic data generation pipeline: sentences are tagged with unique XML identifiers, processed by a teacher LLM with chain-of-thought prompting, verified for citation accuracy, and filtered for quality.
• Figure 3: Token distribution of training documents. The long-tail distribution reflects diverse document lengths (truncated at 50K for visibility; maximum is 179K tokens).
• Figure 4: Comparison of baseline vs sui-1 outputs on the same input. Baselines produce generic summaries without citations; sui-1 generates specific claims with verifiable XML tags.
• Figure 5: Example sui-1 output showing first-person reasoning about summary structure, followed by the summary with inline citations. Interactive demo: https://huggingface.co/spaces/ellamind/sui-demo