DnDScore: Decontextualization and Decomposition for Factuality Verification in Long-Form Text Generation

TL;DR

This work investigates the interaction between decomposition and decontextualization in factuality verification for long-form text. It introduces DnD, a joint prompt-based method that yields paired subclaims and their decontextualized forms, and proposes DnDScore, a verification metric that uses context to assess subclaims. Through experiments on biography data, the authors show that the choice of decomposition and decontextualization strategy significantly affects factuality scores, and that context-aware verification can both improve and complicate verification outcomes. The study demonstrates that DnDScore provides a more robust, context-sensitive approach to factuality verification, with practical implications for building more reliable long-form LLM outputs.

Abstract

The decompose-then-verify strategy for verification of Large Language Model (LLM) generations decomposes claims that are then independently verified. Decontextualization augments text (claims) to ensure it can be verified outside of the original context, enabling reliable verification. While decomposition and decontextualization have been explored independently, their interactions in a complete system have not been investigated. Their conflicting purposes can create tensions: decomposition isolates atomic facts while decontextualization inserts relevant information. Furthermore, a decontextualized subclaim presents a challenge to the verification step: what part of the augmented text should be verified as it now contains multiple atomic facts? We conduct an evaluation of different decomposition, decontextualization, and verification strategies and find that the choice of strategy matters in the resulting factuality scores. Additionally, we introduce DnDScore, a decontextualization aware verification method which validates subclaims in the context of contextual information.

Figures (1)

• Figure 1: Current claim verification methods evaluate subclaims out of context, but adding this context into the claim verification pipeline is not trivial. We introduce DnDScore, a method that evaluates subclaims given the decontextualized claim or another context to help verify.