Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Revealing economic facts: LLMs know more than they say

Marcus Buckmann, Quynh Anh Nguyen, Edward Hill

TL;DR

This study shows that hidden states (embeddings) of open-source LLMs contain richer economic information than the models' textual outputs. A simple ridge regression on embeddings (LME) frequently outperforms the LLM's own answers for estimating county- and firm-level statistics, especially for rarer variables, and requires only a small number of labelled examples. The authors also introduce transfer-learning approaches that estimate target variables without labels by leveraging embeddings from other variables or noisy text labels, achieving robust improvements. Additionally, embeddings prove useful for data-processing tasks such as imputation and super-resolution, enhancing practical data pipelines without requiring knowledge of the target statistics during training.

Abstract

We investigate whether the hidden states of large language models (LLMs) can be used to estimate and impute economic and financial statistics. Focusing on county-level (e.g. unemployment) and firm-level (e.g. total assets) variables, we show that a simple linear model trained on the hidden states of open-source LLMs outperforms the models' text outputs. This suggests that hidden states capture richer economic information than the responses of the LLMs reveal directly. A learning curve analysis indicates that only a few dozen labelled examples are sufficient for training. We also propose a transfer learning method that improves estimation accuracy without requiring any labelled data for the target variable. Finally, we demonstrate the practical utility of hidden-state representations in super-resolution and data imputation tasks.

Revealing economic facts: LLMs know more than they say

TL;DR

This study shows that hidden states (embeddings) of open-source LLMs contain richer economic information than the models' textual outputs. A simple ridge regression on embeddings (LME) frequently outperforms the LLM's own answers for estimating county- and firm-level statistics, especially for rarer variables, and requires only a small number of labelled examples. The authors also introduce transfer-learning approaches that estimate target variables without labels by leveraging embeddings from other variables or noisy text labels, achieving robust improvements. Additionally, embeddings prove useful for data-processing tasks such as imputation and super-resolution, enhancing practical data pipelines without requiring knowledge of the target statistics during training.

Abstract

We investigate whether the hidden states of large language models (LLMs) can be used to estimate and impute economic and financial statistics. Focusing on county-level (e.g. unemployment) and firm-level (e.g. total assets) variables, we show that a simple linear model trained on the hidden states of open-source LLMs outperforms the models' text outputs. This suggests that hidden states capture richer economic information than the responses of the LLMs reveal directly. A learning curve analysis indicates that only a few dozen labelled examples are sufficient for training. We also propose a transfer learning method that improves estimation accuracy without requiring any labelled data for the target variable. Finally, we demonstrate the practical utility of hidden-state representations in super-resolution and data imputation tasks.
Paper Structure (31 sections, 16 figures, 4 tables)

This paper contains 31 sections, 16 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Literature review
  3. Methodology
  4. Datasets
  5. Experimental set-up
  6. Linear model on embeddings (LME)
  7. Data transformations and performance metrics
  8. Using LLM embeddings to infer statistics
  9. Cross-validation
  10. Robustness
  11. Comparing base models
  12. LME vs. a reasoning model
  13. Learning curves
  14. Transfer learning
  15. Learning from embeddings of other variables
  16. ...and 16 more sections

Figures (16)

  • Figure 1: Cross-validation performance. The variable labels are shortened. The full variable names used when querying the LLM are shown in Table \ref{['tab:tab_cv_all']}.
  • Figure 2: Comparing actual values (ground truth) to the predicted values by LME (top row) and the text output (bottom row).
  • Figure 3: Performance of LLMs of different sizes.
  • Figure 4: Cross-validation performance comparing reasoning model Qwen QwQ (text output) to Qwen 2.5 (text output + LME).
  • Figure 5: Learning curve analysis.
  • ...and 11 more figures