Mitigating Content Effects on Reasoning in Language Models through Fine-Grained Activation Steering

TL;DR

This work tackles the bias where content plausibility affects formal reasoning in large language models. It introduces a controlled syllogistic dataset and uses activation steering to intervene at test time, first with static contrastive methods and then with dynamic, fine-grained CAST and K-CAST approaches. The results show that contrastive steering improves formal reasoning in many models, while dynamic, kNN-based conditional steering enables substantial gains (up to about 15 percentage points) on otherwise unresponsive models and maintains robustness to prompt variations with limited side effects on language modeling. The findings suggest activation-level interventions can be a scalable strategy to enhance the reliability and generalization of LLMs in rigorous reasoning tasks, with promising implications for out-of-distribution and multilingual settings.

Abstract

Large language models (LLMs) frequently demonstrate reasoning limitations, often conflating content plausibility (i.e., material inference) with logical validity (i.e., formal inference). This can result in biased inferences, where plausible arguments are incorrectly deemed logically valid or vice versa. Mitigating this limitation is critical, as it undermines the trustworthiness and generalizability of LLMs in applications that demand rigorous logical consistency. This paper investigates the problem of mitigating content biases on formal reasoning through activation steering. Specifically, we curate a controlled syllogistic reasoning dataset to disentangle formal validity from content plausibility. After localising the layers responsible for formal and material inference, we investigate contrastive activation steering methods for test-time interventions. An extensive empirical analysis on different LLMs reveals that contrastive steering consistently supports linear control over content biases. However, we observe that a static approach is insufficient for improving all the tested models. We then leverage the possibility to control content effects by dynamically determining the value of the steering parameters via fine-grained conditional methods. We found that conditional steering is effective on unresponsive models, achieving up to 15% absolute improvement in formal reasoning accuracy with a newly introduced kNN-based method (K-CAST). Finally, additional experiments reveal that steering for content effects is robust to prompt variations, incurs minimal side effects on language modeling capabilities, and can partially generalize to out-of-distribution reasoning tasks. Practically, this paper demonstrates that activation-level interventions can offer a scalable strategy for enhancing the robustness of LLMs, contributing towards more systematic and unbiased formal reasoning.

Figures (8)

• Figure 1: Overview of our methodology for mitigating content effects on formal reasoning via activation steering. We first curate a controlled syllogistic reasoning dataset designed to disentangle formal validity from content plausibility. Subsequently, after localising the layers mostly responsible for formal and material inference through probing, we investigate static and conditional contrastive steering methods for test-time interventions to debias models' behaviour.
• Figure 2: Linear probing results for formal validity on Llama-3.1 8b. The probing experiments reveal that the information for both validity and plausibility is maximally localised in later layers, peaking at the last third quarter of the layers in the residual stream across different LLMs (see Appendix).
• Figure 3: (Top) Example of effective (Llama 1b) and ineffective (Llama 3b) contrastive steering with static values of $\alpha$. (Bottom) Impact of conditional activation steering (K-CAST) on Llama 3b. Contrary to static steering, K-CAST leads to a significant increase in accuracy for Llama 3b (i.e., up to 15%) while substantially reducing content effect.
• Figure 4: Robustness of steering to prompt variations on Llama 1b (i.e., ACC, CE, and ACC/CE). The results reveal that, despite some noise deriving from perturbations applied at test time, the overall effectiveness of steering remains unaltered.
• Figure 5: Linear Probing Results for different models. In general, the probing results suggest that the information about validity and plausibility is encoded in the second half of the layers, with a peak in the third quarter (predominantly in the residual stream).