Navigating through the hidden embedding space: steering LLMs to improve mental health assessment

TL;DR

This work tackles the gap where compact LLMs underperform in mental health (MH) assessment compared to larger models. It introduces an inference-time steering technique that applies a linear shift to activations at a mid-layer via a steering vector $v_s$ with strength $\lambda$, calibrated through a principled, data-driven criterion. The approach improves zero-shot relevance prediction for BDI-II items and enhances automated BDI-II questionnaire completion from Reddit histories, achieving competitive results with significantly larger models and without retraining. Collectively, the findings suggest steering vectors as a scalable, computationally efficient method to adapt compact LLMs for domain-specific MH tasks, with implications for safe, resource-conscious deployment.

Abstract

The rapid evolution of Large Language Models (LLMs) is transforming AI, opening new opportunities in sensitive and high-impact areas such as Mental Health (MH). Yet, despite these advancements, recent evidence reveals that smaller-scale models still struggle to deliver optimal performance in domain-specific applications. In this study, we present a cost-efficient yet powerful approach to improve MH assessment capabilities of an LLM, without relying on any computationally intensive techniques. Our lightweight method consists of a linear transformation applied to a specific layer's activations, leveraging steering vectors to guide the model's output. Remarkably, this intervention enables the model to achieve improved results across two distinct tasks: (1) identifying whether a Reddit post is useful for detecting the presence or absence of depressive symptoms (relevance prediction task), and (2) completing a standardized psychological screening questionnaire for depression based on users' Reddit post history (questionnaire completion task). Results highlight the untapped potential of steering mechanisms as computationally efficient tools for LLMs' MH domain adaptation.

Figures (2)

• Figure 1: Overview of the proposed steering-based methodology for mental health assessment using LLMs. (A) In the DepreSym dataset, Reddit posts annotated for their relevance to specific BDI-II items are used to derive steering vectors. These vectors represent the directional difference between positive and negative representations in the embedding space and are applied as linear transformations to guide the model’s internal activations. The examples shown in the Figure are generated for illustration purposes and do not correspond to actual dataset samples. (B) In the downstream eRisk 2021 task, the steered LLM leverages these calibrated representations to complete BDI-II questionnaires directly from users’ Reddit post histories in a zero-shot setting, enabling symptom- and severity-level inference without any additional training.
• Figure 2: (A) Confusion matrices showing classification results across different steering strengths ($\lambda = -2, -1, \lambda^*, 0, 1, 2$) on the DepreSym test set. (B) Distribution of logits for positive observations under different steering configurations for indecisiveness and sadness classification tasks. (C) Prompt templates used for relevance prediction and BDI-II questionnaire completion tasks. (D) DCHR performance comparison across different steering strengths and distribution of item-specific optimal values $\lambda_j^*$ (21 in total).