Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On the Similarity of Circuits across Languages: a Case Study on the Subject-verb Agreement Task

Javier Ferrando, Marta R. Costa-jussà

TL;DR

It is discovered that both circuits implemented by Gemma 2B for solving the subject-verb agreement task across two different languages, English and Spanish, are highly consistent, being mainly driven by a particular attention head writing a `subject number' signal to the last residual stream, which is read by a small set of neurons in the final MLPs.

Abstract

Several algorithms implemented by language models have recently been successfully reversed-engineered. However, these findings have been concentrated on specific tasks and models, leaving it unclear how universal circuits are across different settings. In this paper, we study the circuits implemented by Gemma 2B for solving the subject-verb agreement task across two different languages, English and Spanish. We discover that both circuits are highly consistent, being mainly driven by a particular attention head writing a `subject number' signal to the last residual stream, which is read by a small set of neurons in the final MLPs. Notably, this subject number signal is represented as a direction in the residual stream space, and is language-independent. We demonstrate that this direction has a causal effect on the model predictions, effectively flipping the Spanish predicted verb number by intervening with the direction found in English. Finally, we present evidence of similar behavior in other models within the Gemma 1 and Gemma 2 families.

On the Similarity of Circuits across Languages: a Case Study on the Subject-verb Agreement Task

TL;DR

It is discovered that both circuits implemented by Gemma 2B for solving the subject-verb agreement task across two different languages, English and Spanish, are highly consistent, being mainly driven by a particular attention head writing a `subject number' signal to the last residual stream, which is read by a small set of neurons in the final MLPs.

Abstract

Several algorithms implemented by language models have recently been successfully reversed-engineered. However, these findings have been concentrated on specific tasks and models, leaving it unclear how universal circuits are across different settings. In this paper, we study the circuits implemented by Gemma 2B for solving the subject-verb agreement task across two different languages, English and Spanish. We discover that both circuits are highly consistent, being mainly driven by a particular attention head writing a `subject number' signal to the last residual stream, which is read by a small set of neurons in the final MLPs. Notably, this subject number signal is represented as a direction in the residual stream space, and is language-independent. We demonstrate that this direction has a causal effect on the model predictions, effectively flipping the Spanish predicted verb number by intervening with the direction found in English. Finally, we present evidence of similar behavior in other models within the Gemma 1 and Gemma 2 families.
Paper Structure (24 sections, 6 equations, 25 figures, 3 tables)

This paper contains 24 sections, 6 equations, 25 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Experimental Setup
  3. Methods
  4. Direct Logit Attribution.
  5. Activation Patching.
  6. English Subject-Verb Agreement Circuit
  7. Locating relevant components and residual stream states.
  8. Analysis of Neurons.
  9. An Attention Head Writes the Subject Number in a Single Direction.
  10. Spanish Subject-Verb Agreement Circuit
  11. Spanish circuit is consistent with the English circuit.
  12. Activation Steering.
  13. Replicating Results in Gemma 7B and Gemma 2 2B
  14. Related Work
  15. Conclusion
  16. ...and 9 more sections

Figures (25)

  • Figure 1: English dataset activation patching results on the logit difference metric on (a) the residual streams (b) attention blocks outputs, (c) MLP outputs, and (d) on attention heads at the last position.
  • Figure 2: Average contribution to the logit difference by each model component.
  • Figure 3: Average contribution to the logit difference by each neuron in MLP13.
  • Figure 4: Dot product of the output of attention head L13H7 and the input weights of neuron 2069 in MLP13.
  • Figure 5: Projections of L13H7 outputs onto the top 2 PCs on English (left) and Spanish (right) dataset.
  • ...and 20 more figures