A Financial Brain Scan of the LLM

TL;DR

This approach can map LLM-generated economic forecasts to concepts such as sentiment, technical analysis, and timing, and compute their relative importance without reducing performance, and is transparent, lightweight, and replicable for empirical research in the social sciences.

Abstract

Emerging techniques in computer science make it possible to "brain scan" large language models (LLMs), identify the plain-English concepts that guide their reasoning, and steer them while holding other factors constant. We show that this approach can map LLM-generated economic forecasts to concepts such as sentiment, technical analysis, and timing, and compute their relative importance without reducing performance. We also show that models can be steered to be more or less risk-averse, optimistic, or pessimistic, which allows researchers to correct or simulate biases. The method is transparent, lightweight, and replicable for empirical research in the social sciences.

Figures (11)

• Figure 1: Illustration: Steering LLM's Risk Aversion We prompt the LLM to allocate $100 between the S&P500 and bonds. We then vary the intensity with which the model is steered to activate the "financial risk" feature (x-axis) and record the resulting share allocated to the S&P500 (y-axis). To reduce variance, the experiment is repeated across 100 random seeds, and we report the averaged outcomes.
• Figure 2: Illustration of SAE's integration with a Transformer model. The residual stream $\mathbf{r}^{(l)}$ is extracted from an intermediate layer of the language model and projected onto a sparse, interpretable representation $\mathbf{z}^{(l)}$. This sparse code is then decoded to reconstruct $\hat{\mathbf{r}}^{(l)}$, which is fed back into the model for subsequent processing.
• Figure 3: Top 5 Contributing Features This figure reports the labels of the five features with the highest contribution to return prediction, obtained by replicating the exercise of chen2022expected. For each feature, the bars report the absolute weights assigned by the forecasting model, averaged across rolling windows.
• Figure 4: Wordclouds These figures display the most frequent words in the feature labels within each cluster. The subcaptions indicate the names assigned to the respective clusters.
• Figure 5: Correlation Between Predictions Across Clusters Pairwise correlations of out-of-sample return predictions from models estimated on individual feature clusters. Blue indicates lower correlations, whereas red denotes higher correlations.