Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

To Copy or Not to Copy: Copying Is Easier to Induce Than Recall

Mehrdad Farahani, Franziska Penzkofer, Richard Johansson

TL;DR

This work tackles how large language models arbitrate between parametric knowledge and contextual information in retrieval-augmented settings. It introduces an arbitration vector, computed from residual-stream centroids that distinguish IC from RC across 27 relations, and demonstrates that additive steering can toggle between Copy$\rightarrow$Recall and Recall$\rightarrow$Copy in two architectures (Gemma2-2B and T5Gemma-2B) and two QA benchmarks. The results reveal a robust asymmetry: copying from context is easier to induce than recalling from memory, with mechanistic analyses showing attention reallocation and MLP-driven updates as key drivers. The findings offer a general, non-fine-tuned control signal for directing information sourcing in RAG-like systems, while acknowledging limitations related to prompting, model scale, and end-to-end retriever integration.

Abstract

Language models used in retrieval-augmented settings must arbitrate between parametric knowledge stored in their weights and contextual information in the prompt. This work presents a mechanistic study of that choice by extracting an \emph{arbitration vector} from model activations on a curated dataset designed to disentangle (i) irrelevant contexts that elicit parametric recall and (ii) relevant but false contexts that elicit copying. The vector is computed as the residual-stream centroid difference between these regimes across 27 relations, and is injected as an additive intervention at selected layers and token spans to steer behavior in two directions: Copy$\rightarrow$Recall (suppressing context use) and Recall$\rightarrow$Copy (inducing the model to copy any token from the context). Experiments on two architectures (decoder-only and encoder/decoder) and two open-domain QA benchmarks show consistent behavior shifts under moderate scaling while monitoring accuracy and fluency. Mechanistic analyses of attention routing, MLP contributions, and layer-wise probability trajectories reveal an asymmetry: inducing copying is an easy ``reactivation'' process that can be triggered at different locations in the input, while restoring recall is a ``suppression'' process that is more fragile and strongly tied to object-token interventions.

To Copy or Not to Copy: Copying Is Easier to Induce Than Recall

TL;DR

This work tackles how large language models arbitrate between parametric knowledge and contextual information in retrieval-augmented settings. It introduces an arbitration vector, computed from residual-stream centroids that distinguish IC from RC across 27 relations, and demonstrates that additive steering can toggle between CopyRecall and RecallCopy in two architectures (Gemma2-2B and T5Gemma-2B) and two QA benchmarks. The results reveal a robust asymmetry: copying from context is easier to induce than recalling from memory, with mechanistic analyses showing attention reallocation and MLP-driven updates as key drivers. The findings offer a general, non-fine-tuned control signal for directing information sourcing in RAG-like systems, while acknowledging limitations related to prompting, model scale, and end-to-end retriever integration.

Abstract

Language models used in retrieval-augmented settings must arbitrate between parametric knowledge stored in their weights and contextual information in the prompt. This work presents a mechanistic study of that choice by extracting an \emph{arbitration vector} from model activations on a curated dataset designed to disentangle (i) irrelevant contexts that elicit parametric recall and (ii) relevant but false contexts that elicit copying. The vector is computed as the residual-stream centroid difference between these regimes across 27 relations, and is injected as an additive intervention at selected layers and token spans to steer behavior in two directions: CopyRecall (suppressing context use) and RecallCopy (inducing the model to copy any token from the context). Experiments on two architectures (decoder-only and encoder/decoder) and two open-domain QA benchmarks show consistent behavior shifts under moderate scaling while monitoring accuracy and fluency. Mechanistic analyses of attention routing, MLP contributions, and layer-wise probability trajectories reveal an asymmetry: inducing copying is an easy ``reactivation'' process that can be triggered at different locations in the input, while restoring recall is a ``suppression'' process that is more fragile and strongly tied to object-token interventions.
Paper Structure (86 sections, 2 equations, 51 figures, 3 tables)

This paper contains 86 sections, 2 equations, 51 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. Models
  4. Prompt Topology
  5. Datasets
  6. Arbitration Dataset
  7. Evaluation Benchmark
  8. Arbitration Vector
  9. Evaluation Framework
  10. Benchmark Selection
  11. Experimental Scenarios
  12. Parametric Steering (Copy$\rightarrow$Recall)
  13. Contextual Steering (Recall$\rightarrow$Copy)
  14. Behavioral Metrics
  15. Performance (EM and F1)
  16. ...and 71 more sections

Figures (51)

  • Figure 1: Comparison of steering within two regimes. (a) Suppression forces the model to ignore context. (b) Reactivation forces the model to copy from context.
  • Figure 2: Layer-wise EM on PopQA. Columns 1,3 show parametric steering ($\alpha = 30.0$) recovering internal knowledge (peaks: 66% and 74%). Columns 2,4 show contextual steering ($\alpha = -3.0$), inducing copying behavior (peaks: 34% and 46%).
  • Figure 3: Layer-wise F1 score (Gemma2-2B, PopQA) of parametric steering (C$\rightarrow$R) on counterfactual object.
  • Figure 4: Layer-wise PPL score (Gemma2-2B, PopQA) of parametric steering (C$\rightarrow$R) on counterfactual object.
  • Figure 5: Layer-wise F1 score (Gemma2-2B, PopQA) of contextual steering (R$\rightarrow$C) on counterfactual object.
  • ...and 46 more figures