Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CBMAS: Cognitive Behavioral Modeling via Activation Steering

Ahmed H. Ismail, Anthony Kuang, Ayo Akinkugbe, Kevin Zhu, Sean O'Brien

TL;DR

CBMAS tackles diagnosing and controlling cognitive biases in LLMs by introducing continuous activation steering and bias trajectory analysis across model depth via $alpha$-sweeps. It builds layer-specific steering directions from contrastive prompts and tracks their effects with Bias Response Curves, computing metrics such as $Delta_logit$ and $Delta_prob$, as well as $KL$ divergence to quantify shifts while monitoring fluency. The key contributions are a practical diagnostic framework, release of contrastive datasets for Sycophancy, Reassurance, Satisficing, and Deference, and empirical demonstrations of tipping points and layer-site dependencies. This work advances cognitive interpretability and provides tools for safer, more controllable LLM deployment.

Abstract

Large language models (LLMs) often encode cognitive behaviors unpredictably across prompts, layers, and contexts, making them difficult to diagnose and control. We present CBMAS, a diagnostic framework for continuous activation steering, which extends cognitive bias analysis from discrete before/after interventions to interpretable trajectories. By combining steering vector construction with dense α-sweeps, logit lens-based bias curves, and layer-site sensitivity analysis, our approach can reveal tipping points where small intervention strengths flip model behavior and show how steering effects evolve across layer depth. We argue that these continuous diagnostics offer a bridge between high-level behavioral evaluation and low-level representational dynamics, contributing to the cognitive interpretability of LLMs. Lastly, we provide a CLI and datasets for various cognitive behaviors at the project repository, https://github.com/shimamooo/CBMAS.

CBMAS: Cognitive Behavioral Modeling via Activation Steering

TL;DR

CBMAS tackles diagnosing and controlling cognitive biases in LLMs by introducing continuous activation steering and bias trajectory analysis across model depth via -sweeps. It builds layer-specific steering directions from contrastive prompts and tracks their effects with Bias Response Curves, computing metrics such as and , as well as divergence to quantify shifts while monitoring fluency. The key contributions are a practical diagnostic framework, release of contrastive datasets for Sycophancy, Reassurance, Satisficing, and Deference, and empirical demonstrations of tipping points and layer-site dependencies. This work advances cognitive interpretability and provides tools for safer, more controllable LLM deployment.

Abstract

Large language models (LLMs) often encode cognitive behaviors unpredictably across prompts, layers, and contexts, making them difficult to diagnose and control. We present CBMAS, a diagnostic framework for continuous activation steering, which extends cognitive bias analysis from discrete before/after interventions to interpretable trajectories. By combining steering vector construction with dense α-sweeps, logit lens-based bias curves, and layer-site sensitivity analysis, our approach can reveal tipping points where small intervention strengths flip model behavior and show how steering effects evolve across layer depth. We argue that these continuous diagnostics offer a bridge between high-level behavioral evaluation and low-level representational dynamics, contributing to the cognitive interpretability of LLMs. Lastly, we provide a CLI and datasets for various cognitive behaviors at the project repository, https://github.com/shimamooo/CBMAS.
Paper Structure (18 sections, 7 equations, 6 figures, 4 tables)

This paper contains 18 sections, 7 equations, 6 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Activation Steering
  4. Mechanistic Interpretability Tools
  5. Bias and Alignment Research
  6. Methodology
  7. Contrastive Prompt Dataset
  8. Steering Vector Construction
  9. Multi-Layer Intervention and Analysis
  10. Bias Response Curve Protocol
  11. Experiment and Results
  12. Continuous Trajectories vs. Binary Snapshots
  13. Tipping points and Fluency
  14. Injection Sites as Causal Leverage Points
  15. Conclusions and Limitations
  16. ...and 3 more sections

Figures (6)

  • Figure 1: A contrastive prompt pair $(p_k^{(A)}, p_k^{(B)})$
  • Figure 2: The generation of a bias vector for a layer $L$, where the green and red vectors correspond to supportive and unsupportive directions, respectively.
  • Figure 3: $\alpha$-sweeps on reassurance vectors (inj $L_0$). $\Delta_{\text{logit}}(\alpha)$ shows a steep positive slope and zero-crossing at $L_1$, which weakens to a shallow slope by $L_{11}$.
  • Figure 4: Rank change overlay shows supportive tokens overtaking at $\alpha \approx 0$.
  • Figure 5: (a) $\Delta_{\text{logit}}$ at $L_6$ rises with $\alpha$ while random/orthogonal controls are flat. (b) KL divergence at $L_{11}$ stays low and symmetric (fluency preserved).
  • ...and 1 more figures