Factuality on Demand: Controlling the Factuality-Informativeness Trade-off in Text Generation

TL;DR

This work tackles the challenge of balancing factuality and informativeness in knowledge-heavy text generation. It introduces Factuality-Controlled Generation (FCG), a framework that conditions outputs on a numeric factuality level $c$ to trade off accuracy and content. The authors develop a synthetic data pipeline that constructs $(x,c,r)$ triplets by removing low-confidence facts to meet $f(r) \geq c$, and they fine-tune a mid-sized model (Mistral-7B-Instruct) on these examples. Empirical results on biography generation show that FCG substantially improves factuality adherence at high $c$ values and shifts the trade-off frontier outward, preserving informativeness where possible. This approach provides a practical, controllable mechanism for customizing LLM outputs to domain-specific factuality requirements, with implications for safer, more reliable knowledge-grounded generation.

Abstract

Large language models (LLMs) encode knowledge with varying degrees of confidence. When responding to queries, models face an inherent trade-off: they can generate responses that are less informative but highly factual, or more informative but potentially less accurate. Different applications demand different balances between informativeness and factuality. We introduce Factuality-Controlled Generation (FCG), a framework that enables users to specify factuality constraints alongside their queries. We propose to evaluate FCG performance on two dimensions: adherence to factuality constraints and response informativeness. We propose to train models on the FCG task using synthetic data, and show that our synthetic training significantly improves models' ability to both respect factuality requirements and maintain informativeness in their outputs.

Figures (3)

• Figure 1: Synthetic data generation pipeline for creating (question, control level, response) pairs. Given a question, we generate an initial response, segment it into facts, score each fact’s confidence, and remove the lowest-confidence facts until the response’s factuality meets the control level $c$. In the prompts, we express $c$as a percentage ($c = x\%$). This yields a filtered response that adheres to the factuality constraint while maximizing informativeness, which is then used for factuality-controlled training (fine-tuning the model).
• Figure 2: Our factuality-controlled training method improves the model's ability to respect factuality constraints: as the target factuality level $c$ increases, the model's outputs become more factual (yellow line). In contrast, the baseline models show no consistent improvement (red and blue lines).
• Figure 3: Factuality vs. informativeness trade-off curves for three methods. No factuality control (NFC) uses the base model with no control instruction. Factuality-controlled inference (FCI) uses the base model with a test-time control instruction but no fine-tuning. Factuality-controlled training (FCG) fine-tunes the model on synthetic (question, control, response) data and applies the same test-time control. FCG achieves a better trade-off than the baselines (higher informativeness at the same factuality), and at $c = 1.0$ it still produces informative responses (23.6% of outputs contain only verified facts), whereas the baselines produce no fully factual outputs.