Natural Language Descriptions of Deep Visual Features

TL;DR

Many deep networks rely on neurons that respond to specific perceptual features, but existing labeling methods cover only a subset of neuron behaviors. MILAN automatically generates open-ended, natural-language descriptions for individual neurons by maximizing pointwise mutual information between descriptions and neuron-activated image regions, using models trained on the milannotations dataset. The authors demonstrate generalization across architectures, datasets, and tasks, and illustrate three practical uses: analyzing feature distributions, auditing for demographically sensitive features in anonymized data, and editing spurious text-related features to improve robustness. This framework provides a scalable, model-agnostic tool for fine-grained neuron interpretability with direct implications for dataset design, model auditing, and controlled editing, while acknowledging annotation noise and domain-shift limitations.

Abstract

Some neurons in deep networks specialize in recognizing highly specific perceptual, structural, or semantic features of inputs. In computer vision, techniques exist for identifying neurons that respond to individual concept categories like colors, textures, and object classes. But these techniques are limited in scope, labeling only a small subset of neurons and behaviors in any network. Is a richer characterization of neuron-level computation possible? We introduce a procedure (called MILAN, for mutual-information-guided linguistic annotation of neurons) that automatically labels neurons with open-ended, compositional, natural language descriptions. Given a neuron, MILAN generates a description by searching for a natural language string that maximizes pointwise mutual information with the image regions in which the neuron is active. MILAN produces fine-grained descriptions that capture categorical, relational, and logical structure in learned features. These descriptions obtain high agreement with human-generated feature descriptions across a diverse set of model architectures and tasks, and can aid in understanding and controlling learned models. We highlight three applications of natural language neuron descriptions. First, we use MILAN for analysis, characterizing the distribution and importance of neurons selective for attribute, category, and relational information in vision models. Second, we use MILAN for auditing, surfacing neurons sensitive to human faces in datasets designed to obscure them. Finally, we use MILAN for editing, improving robustness in an image classifier by deleting neurons sensitive to text features spuriously correlated with class labels.

Figures (15)

• Figure 1: (a) We aim to generate natural language descriptions of individual neurons in deep networks. (b) We first represent each neuron via an exemplar set of input regions that activate it. (c) In parallel, we collect a dataset of fine-grained human descriptions of image regions, and use these to train a model of $p(\textrm{description} \mid \text{exemplars})$ and $p(\textrm{description})$. (d) Using these models, we search for a description that has high pointwise mutual information with the exemplars, ultimately generating highly specific neuron annotations.
• Figure 2: Examples of milan descriptions on the generalization tasks described in \ref{['sec:generalization']}. Even highly specific labels (like the top boundaries of horizontal objects) can be predicted for neurons in new networks. Failure modes include semantic errors, e.g. milan misses the cupcakes in the dog faces and cupcakes neuron.
• Figure 3: Examples of milan failures. Failure modes include incorrect generalization (top), vague descriptions for concepts not seen in the training set (middle), and mistaking the context for the highlighted regions (bottom).
• Figure 4: ResNet18 accuracy on the ImageNet validation set as units are ablated (left, middle), and distribution of neurons matching syntactic and structural criteria in each layer (right). In each configuration, neurons are scored according to a property of their generated description (e.g., number of nouns/words in description, etc.), sorted based on their score, and ablated in that order. Neurons described with adjectives appear crucial for good performance, while neurons described with very different words (measured by word embedding difference; max word diff.) appear less important for good performance. Adjective-selective neurons are most prevalent in early layers, while neurons with large semantic differences are more prevalent in late ones.
• Figure 5: Change in # of face neurons found by milan (each pair of points is one model architecture). Blurring reduces, but does not eliminate, units selective for unblurred faces.

Theorems & Definitions (2)

• Definition 1
• Definition 2