Behavioral Analysis of Information Salience in Large Language Models

TL;DR

This work tackles the unclear notion of information salience learned by large language models during summarization. It introduces an explainable framework that uses length-controlled summarization as a behavioral probe and questions-under-discussion (QUDs) to derive an answerability-based salience proxy, enabling interpretable content-selection analysis across 13 models and four datasets. The study finds that LLMs harbor a nuanced, hierarchical salience that is consistent across model families and sizes, yet inaccessible to direct introspection and only weakly aligned with human salience. These results have practical implications for prompting, evaluation, and training strategies, suggesting that model-driven salience signals may require explicit alignment or training signals to match human expectations. The framework opens avenues for diagnosing content selection in synthesis tasks and exploring salience emergence during model training across domains.

Abstract

Large Language Models (LLMs) excel at text summarization, a task that requires models to select content based on its importance. However, the exact notion of salience that LLMs have internalized remains unclear. To bridge this gap, we introduce an explainable framework to systematically derive and investigate information salience in LLMs through their summarization behavior. Using length-controlled summarization as a behavioral probe into the content selection process, and tracing the answerability of Questions Under Discussion throughout, we derive a proxy for how models prioritize information. Our experiments on 13 models across four datasets reveal that LLMs have a nuanced, hierarchical notion of salience, generally consistent across model families and sizes. While models show highly consistent behavior and hence salience patterns, this notion of salience cannot be accessed through introspection, and only weakly correlates with human perceptions of information salience.

Figures (18)

• Figure 1: Framework overview, conceptualizing content salience as question answerability. Left: Given a corpus, we derive questions that are typically answered in summaries. Length-controlled summarization acts as a probe into the content-selection process of LLMs. Question paraphrases are clustered by semantic intent. Middle: Answerability is calculated as the fraction of document-answer claims entailed by the summary. Right: The content salience map tracks answerability at each summary length. More salient questions remain answerable even in shorter summaries.
• Figure 2: Corpus-level content salience map for RCT summaries by four methods (continued in \ref{['fig:salience-pubmed-full']}).
• Figure 3: Do LLMs share a similar notion of salience? Heatmaps show agreement of content-selection at the atomic-claim level (Krippendorff's $\alpha$). Dashed bounding boxes indicate models of the same family. The diagonal shows self-agreement over multiple generations. Top row: RCT, Bottom row: CL.
• Figure 4: Agreement with GPT-4o-mini, averaged over all datasets and summary lengths.
• Figure 5: Distribution of target length ratios over all generated summaries (aggregating lengths and datasets).