Identifying and Controlling Important Neurons in Neural Machine Translation

TL;DR

This work shows that individual neurons in neural machine translation carry linguistically meaningful information and are not entirely distributed. It introduces unsupervised cross-model correlation methods (MaxCorr, MinCorr, LinReg, SVCCA) to identify important neurons without annotations, and validates their importance via erasure and lightweight verification. The authors demonstrate linguistically interpretable neurons (e.g., for parentheses, tense) and show that activating or suppressing specific neurons can controllably affect translations, including tense, number, and gender, with varying success. The results have implications for bias mitigation, interpretability, and targeted control in MT, and the framework is adaptable to other neural tasks and architectures.

Abstract

Neural machine translation (NMT) models learn representations containing substantial linguistic information. However, it is not clear if such information is fully distributed or if some of it can be attributed to individual neurons. We develop unsupervised methods for discovering important neurons in NMT models. Our methods rely on the intuition that different models learn similar properties, and do not require any costly external supervision. We show experimentally that translation quality depends on the discovered neurons, and find that many of them capture common linguistic phenomena. Finally, we show how to control NMT translations in predictable ways, by modifying activations of individual neurons.

Figures (9)

• Figure 1: An illustration of the correlation methods methods, showing how to compute the score for one neuron using each of the methods. Here the number of models is $M=3$.
• Figure 2: Erasing neurons (or SVCCA directions) from the top and bottom of the list of most important neurons (directions) ranked by different unsupervised methods, in an English-Spanish model.
• Figure 3: Erasing neurons from the top or bottom of the MaxCorr ranking in three language pairs.
• Figure 4: Success rates and BLEU scores for controlling NMT by modifying neuron activations.
• Figure 5: Visualization of a neuron from an English-Spanish model that activates positively (red color) on the first word in the noun phrase and negatively (blue) on the following words.