Atomic Literary Styling: Mechanistic Manipulation of Prose Generation in Neural Language Models

TL;DR

This work addresses how internal neuron activations relate to literary style in large language models by applying mechanistic interpretability to GPT-2 Medium on Bartleby, the Scrivener. It identifies a large set of discriminative neurons ($27{,}122$ with $p<0.05$) and tests causality through ablations, revealing a causality gap where removing such neurons often improves rather than worsens output style. It further demonstrates steering techniques (additive, multiplicative, and clamping) and analyzes the relationship between observed activations and generated prose, showing that correlation does not imply causation in generation. The findings challenge assumptions in interpretability, offer a practical framework for causal probing of internal representations, and point to broader implications for controllable AI and alignment in creative text generation.

Abstract

We present a mechanistic analysis of literary style in GPT-2, identifying individual neurons that discriminate between exemplary prose and rigid AI-generated text. Using Herman Melville's Bartleby, the Scrivener as a corpus, we extract activation patterns from 355 million parameters across 32,768 neurons in late layers. We find 27,122 statistically significant discriminative neurons ($p < 0.05$), with effect sizes up to $|d| = 1.4$. Through systematic ablation studies, we discover a paradoxical result: while these neurons correlate with literary text during analysis, removing them often improves rather than degrades generated prose quality. Specifically, ablating 50 high-discriminating neurons yields a 25.7% improvement in literary style metrics. This demonstrates a critical gap between observational correlation and causal necessity in neural networks. Our findings challenge the assumption that neurons which activate on desirable inputs will produce those outputs during generation, with implications for mechanistic interpretability research and AI alignment.

Figures (3)

• Figure 1: Distribution of Cohen's $d$ effect sizes across all 32,768 analyzed neurons in layers 16-23. The distribution is approximately Gaussian centered near zero, with the strongest discriminating neurons reaching $|d| \approx 1.4$.
• Figure 2: Logit lens analysis tracking prediction evolution across layers. Logit values build steadily through early layers, peak around layers 17-18, then decline. Prediction rank shows the inverse pattern, suggesting mid-to-late layers perform critical refinement based on stylistic context.
• Figure 3: Ablation study results. Left: Individual neuron effects showing mixed results (red bars indicate degradation, green bars indicate improvement). Right: Cumulative ablation curve showing that removing more neurons generally improves literary quality, with style score rising from 0.793 to 0.997 when 50 neurons are ablated.