Efficient Representations are Controllable Representations

TL;DR

The paper addresses controllability of LLMs by introducing a minimal, architecture-agnostic mechanism: install $16$ binary feature flags in a $D=3072$ residual-dimension space of a $3.8$B Transformer, train with a two-stage curriculum to both produce and rely on these flags, and demonstrate that the model naturally consolidates around these fixed-location signals. Training combines an auxiliary position loss with standard language modeling, yielding flags that not only reflect feature usage but also steer generation at inference time when forced, even overriding input semantics. The results show the flags become genuine internal features, the model’s outputs can be directed via controlled flag settings, and other representations are eroded under efficiency pressure, all with only a small perplexity cost at $16$ fenced dims. This work reframes controllability as a consequence of representational pressure under capacity constraints, suggesting a general principle: supply reliable signals at fixed locations, and the model will consolidate around them, enabling writable, interpretable control without architectural changes.

Abstract

What is the most brute-force way to install interpretable, controllable features into a model's activations? Controlling how LLMs internally represent concepts typically requires sophisticated methods to first identify, then intervene on the model's existing feature geometry. We bypass all of this. We finetune an LLM with a simple auxiliary loss, training 16 of its 3072 residual stream dimensions to be inert interpretability flags that simply indicate what concepts are required for generation. The model reorganizes around them anyway, learning to rely on these flags during actual generation tasks. As a result, these inert flags become genuine internal features: interpretable control switches that allow us to steer generation at inference time. Why does this work? When a feature is reliably supplied at a fixed location, gradient descent gradually eliminates redundant encodings elsewhere, and the model erodes its own alternative representations. A model's efficiency pressure is a lever - exploitable to induce interpretable, controllable representations.

Figures (7)

• Figure 1: The residual stream before training. Hidden state at layer $k{=}10$ for the input "Where should I take my dog hiking?" Dimensions 2970--3072 shown; black rectangles mark the fenced regions that will be designated as feature flags. Colors represent activation values. No structure is visible --- fenced regions are indistinguishable from their surroundings.
• Figure 2: The residual stream after training. Same layer, same input as \ref{['fig:hk10_before']}. The animals and dogs regions activate sharply once the model begins processing dog-related content; unrelated regions remain near zero. The installed dimensions now carry interpretable signal that was absent before training.
• Figure 3: Overwriting 16 dimensions steers generation. Four completions of the same prompt under different forced classifications. With no intervention, the model produces a generic story. Forcing animals + dogs yields a dog story. Forcing animals on and dogs off yields an animal story with no dogs. Forcing programming + animals + dogs yields a story about a dog learning to code.
• Figure 4: Synthetic training examples. Each example is assigned a subset of target features. Feature engagement ranges from explicit (A, B) to implicit --- example D requires knowledge of dogs without ever mentioning the word.
• Figure 5: Training dynamics. (a) The first 5k steps show no decline (Stage 1: manual injection). Loss drops sharply once Stage 2 begins. (b) Later layers converge to low loss; early layers plateau, consistent with not yet having derived the relevant features.