Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Model Tells Itself Where to Attend: Faithfulness Meets Automatic Attention Steering

Qingru Zhang, Xiaodong Yu, Chandan Singh, Xiaodong Liu, Liyuan Liu, Jianfeng Gao, Tuo Zhao, Dan Roth, Hao Cheng

TL;DR

AutoPASTA tackles contextual faithfulness in open-book QA by automatically identifying key contextual sentences and explicitly steering a model’s attention to them at inference time, avoiding parameter updates or human annotations. It combines iterative prompting to discover evidence with a targeted attention-steering mechanism and a coarse-to-fine head profiling strategy to efficiently select effective attention heads. Across Natural Questions and HotpotQA, evaluated on Vicuna-7B and LLAMA-3 families, AutoPASTA yields consistent improvements over direct and iterative prompting, including notable cross-domain generalization. The approach improves reliability when processing long or noisy contexts and advances inference-time fidelity in open-domain reasoning tasks.

Abstract

Large language models (LLMs) have demonstrated remarkable performance across various real-world tasks. However, they often struggle to fully comprehend and effectively utilize their input contexts, resulting in responses that are unfaithful or hallucinated. This difficulty increases for contexts that are long or contain distracting information, which can divert LLMs from fully capturing essential evidence. To address this issue, many works use prompting to help LLMs utilize contextual information more faithfully. For instance, iterative prompting highlights key information in two steps that first ask the LLM to identify important pieces of context and then derive answers accordingly. However, prompting methods are constrained to highlighting key information implicitly in token space, which is often insufficient to fully steer the model's attention. To improve model faithfulness more reliably, we propose AutoPASTA, a method that automatically identifies key contextual information and explicitly highlights it by steering an LLM's attention scores. Like prompting, AutoPASTA is applied at inference time and does not require changing any model parameters. Our experiments on open-book QA demonstrate that AutoPASTA effectively enables models to grasp essential contextual information, leading to substantially improved model faithfulness and performance, e.g., an average improvement of 7.95% for LLAMA3-70B-Instruct. Code will be publicly available at https://github.com/QingruZhang/AutoPASTA .

Model Tells Itself Where to Attend: Faithfulness Meets Automatic Attention Steering

TL;DR

AutoPASTA tackles contextual faithfulness in open-book QA by automatically identifying key contextual sentences and explicitly steering a model’s attention to them at inference time, avoiding parameter updates or human annotations. It combines iterative prompting to discover evidence with a targeted attention-steering mechanism and a coarse-to-fine head profiling strategy to efficiently select effective attention heads. Across Natural Questions and HotpotQA, evaluated on Vicuna-7B and LLAMA-3 families, AutoPASTA yields consistent improvements over direct and iterative prompting, including notable cross-domain generalization. The approach improves reliability when processing long or noisy contexts and advances inference-time fidelity in open-domain reasoning tasks.

Abstract

Large language models (LLMs) have demonstrated remarkable performance across various real-world tasks. However, they often struggle to fully comprehend and effectively utilize their input contexts, resulting in responses that are unfaithful or hallucinated. This difficulty increases for contexts that are long or contain distracting information, which can divert LLMs from fully capturing essential evidence. To address this issue, many works use prompting to help LLMs utilize contextual information more faithfully. For instance, iterative prompting highlights key information in two steps that first ask the LLM to identify important pieces of context and then derive answers accordingly. However, prompting methods are constrained to highlighting key information implicitly in token space, which is often insufficient to fully steer the model's attention. To improve model faithfulness more reliably, we propose AutoPASTA, a method that automatically identifies key contextual information and explicitly highlights it by steering an LLM's attention scores. Like prompting, AutoPASTA is applied at inference time and does not require changing any model parameters. Our experiments on open-book QA demonstrate that AutoPASTA effectively enables models to grasp essential contextual information, leading to substantially improved model faithfulness and performance, e.g., an average improvement of 7.95% for LLAMA3-70B-Instruct. Code will be publicly available at https://github.com/QingruZhang/AutoPASTA .
Paper Structure (33 sections, 7 equations, 2 figures, 7 tables, 1 algorithm)

This paper contains 33 sections, 7 equations, 2 figures, 7 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background
  3. Problem description.
  4. Multi-head attention.
  5. Post-hoc attention steering.
  6. Method
  7. Automatic Contextual Highlighting
  8. Coarse-to-fine Model Profiling
  9. Experiments
  10. Experimental Setup
  11. Datasets.
  12. Implementation Details.
  13. AutoPASTA Settings.
  14. Coarse-to-fine Model Profiling.
  15. Baselines.
  16. ...and 18 more sections

Figures (2)

  • Figure 1: The illustration of AutoPASTA and alternative methods given a running example. Responses by Vicuna-7B are shown in red square where Authority is the label. Prompting methods (direct and iterative prompting) fail to guide a model to derive correct answers while AutoPASTA successfully steers it to answer correctly by explicitly highlighting identified key parts.
  • Figure 2: Ablation study of AutoPASTA performance when steering different numbers of heads (\ref{['fig:num_head_vicuna_hotpotQA']} and \ref{['fig:num_head_llama3-8b']}) and setting different $\delta$ (\ref{['fig:alpha_llama3-8b']}). Dashed line in red refers to the baseline performance of direct prompting.