NewsEdits 2.0: Learning the Intentions Behind Updating News

TL;DR

NewsEdits 2.0 advances the study of evolving news by introducing a formal edit-intention taxonomy and annotating thousands of revision pairs to learn patterns behind factual updates. It demonstrates that a LongFormer-based LED model can predict when facts in older drafts will change, revealing linguistically grounded cues such as temporality and statistics, and that this prediction can meaningfully improve LLM abstention in RealTimeQA-like tasks. The work further shows that silver-labeled data enable scalable prediction and that high-precision subsets yield substantially better performance than baseline human-level expectations, providing practical benefits for dynamic information queries. Collectively, the approach offers a path toward more reliable AI-assisted news consumption by identifying when information is likely to be outdated and guiding cautious responses in QA systems.

Abstract

As events progress, news articles often update with new information: if we are not cautious, we risk propagating outdated facts. In this work, we hypothesize that linguistic features indicate factual fluidity, and that we can predict which facts in a news article will update using solely the text of a news article (i.e. not external resources like search engines). We test this hypothesis, first, by isolating fact-updates in large news revisions corpora. News articles may update for many reasons (e.g. factual, stylistic, narrative). We introduce the NewsEdits 2.0 taxonomy, an edit-intentions schema that separates fact updates from stylistic and narrative updates in news writing. We annotate over 9,200 pairs of sentence revisions and train high-scoring ensemble models to apply this schema. Then, taking a large dataset of silver-labeled pairs, we show that we can predict when facts will update in older article drafts with high precision. Finally, to demonstrate the usefulness of these findings, we construct a language model question asking (LLM-QA) abstention task. We wish the LLM to abstain from answering questions when information is likely to become outdated. Using our predictions, we show, LLM absention reaches near oracle levels of accuracy.

Figures (8)

• Figure 1: Updates can occur for many different reasons. Shown here, we identify factual updates (e.g. "Event Update" between 1-1), stylistic updates (e.g. "Style-Guide" between 2-3) and narrative updates (e.g. "Add Background" for sentence addition 2).
• Figure 2: Overall paper flow. In Part 1 of our paper, we develop an edits-intention scheme to describe news edits and train models to apply this schema to existing news revision corpora spangher2022newsedits. In Part 2, we use these models to silver-label a large corpus and ask how well we can predict whether a sentence will factually update. In Part 3, we show these predictions can be beneficial for increasing abstention rates during LLM-QA.
• Figure 3: NewsEdits 2.0: Edit-Intentions Schema categories and their subcategories. In this work, we focus mainly on the Factual Edit category. See Appendix \ref{['apx:annotation_guidelinnes']} for definitions for all categories.
• Figure 4: Performance of Fact-update model increases as we increasingly focus on a pool of documents that are categorized as high-likelihood under the top-performing LED model (in Table \ref{['tab:led_results']}). In other words, the model truly shines in the high-precision, high-probability realm.
• Figure 5: The portion of annotation tasks assigned to each worker.