Attribution Gradients: Incrementally Unfolding Citations for Critical Examination of Attributed AI Answers

TL;DR

This work addresses the challenge of verifying attributed AI answers by introducing attribution gradients, a design that incrementally unfolds context from generated claims to contextualized evidence across sources. The authors implement a research prototype built on OpenSciLM and a citation graph pipeline, enabling decomposition of sentences into atomic claims, extraction of supporting and contradicting excerpts (including second-degree references), and in-situ access to source passages with contextual explanations. A within-subject usability study (n=20) shows attribution gradients increase engagement with sources and yield higher-quality revisions, with more facts added and more corrections made, albeit with some misclassifications that do not propagate to final outputs. The results suggest attribution gradients can improve sensemaking and critical examination of AI-generated science answers, offering a practical path toward more transparent, verifiable AI-assisted inquiry in scholarly contexts.

Abstract

AI question answering systems increasingly generate responses with attributions to sources. However, the task of verifying the actual content of these attributions is in most cases impractical. In this paper, we present attribution gradients as a solution. Attribution gradients provide integrated, incremental affordances for diving into an attributed passage. A user can decompose a sentence of an answer into its claims. For each claim, the user can view supporting and contradictory excerpts mined from sources. Those excerpts serve as clickable conduits into the source (in our application, scientific papers). When evidence itself contains more citations, the UI unpacks the evidence into excerpts from the cited sources. These features of attribution gradients facilitate concurrent interconnections among answer, claim, excerpt, and context. In a usability study, we observed greater engagement with sources and richer revision in a task where participants revised an attributed AI answer with attribution gradients and a baseline.

Figures (8)

• Figure 1: Attribution gradients. Users begin with a claim in the generated answer (1) and see color-coded pieces of evidence (2): black indicates direct support; red, direct contradiction; gray, references to papers that support the claim; and pink, references to papers that contradict the claim. They can then explore evidence excerpts (3) that might include citations to secondary references (4). The interface also allows users to view these excerpts in the original PDF (6) alongside an explanation of how the evidence relates to the claim (5). This gradated unfolding of context helps users contextualize and verify each claim.
• Figure 2: Inspecting the first sentence of an AI-generated answer. When the users clicks the button to open evidence for the sentence, the interface decomposes the sentence into atomic claims and displays these claims below the generated sentence.
• Figure 3: Exploring different types of evidence for the claim. When the user clicks on a claim, the interface displays pieces of evidence that relate to the claim. There are four kinds of evidence (from left to right): evidence from retrieved papers that directly support the claim are colored in black; evidence that references other papers that support the claim are colored in gray; evidence that references other papers that directly contradict the claim are colored pink; and evidence from retrieved papers that directly contradict the claim are colored in red. Users can click on a tab of evidence type to filter down to that kind of support---here, only first-degree support is displayed. For visual succinctness, some excerpts are elided from the list above with a jagged line.
• Figure 4: Viewing passages in context. When a user clicks to jump to the source for a piece of evidence, it scrolls to that evidence in a source viewer on the right side of the screen. The evidence excerpt is highlighted. All other extracted passages of evidence are highlighted in a less salient color. The interface also shows "context for highlighted passage," or a brief description meant to help relate the passage to the claim when the excerpt is read out of context.
• Figure 5: Unraveling citations. If a piece of second-degree evidence cites another source for its evidence, users can sometimes unravel the citation to that source. After clicking the button to expand nested citations, excerpts from the cited source are shown that pertain to the claim. In this case, evidence from citation "[118]" is shown that more precisely characterizes the fine-tuned models and LLMs compared in the cited study.