Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Meta-Models: An Architecture for Decoding LLM Behaviors Through Interpreted Embeddings and Natural Language

Anthony Costarelli, Mat Allen, Severin Field

TL;DR

This work investigates meta-models-an architecture using a meta-model that takes activations from an input-model and answers natural language questions about the input-model's behaviors and shows that meta-models generalize well to out-of-distribution tasks.

Abstract

As Large Language Models (LLMs) become increasingly integrated into our daily lives, the potential harms from deceptive behavior underlie the need for faithfully interpreting their decision-making. While traditional probing methods have shown some effectiveness, they remain best for narrowly scoped tasks while more comprehensive explanations are still necessary. To this end, we investigate meta-models-an architecture using a "meta-model" that takes activations from an "input-model" and answers natural language questions about the input-model's behaviors. We evaluate the meta-model's ability to generalize by training them on selected task types and assessing their out-of-distribution performance in deceptive scenarios. Our findings show that meta-models generalize well to out-of-distribution tasks and point towards opportunities for future research in this area. Our code is available at https://github.com/acostarelli/meta-models-public .

Meta-Models: An Architecture for Decoding LLM Behaviors Through Interpreted Embeddings and Natural Language

TL;DR

This work investigates meta-models-an architecture using a meta-model that takes activations from an input-model and answers natural language questions about the input-model's behaviors and shows that meta-models generalize well to out-of-distribution tasks.

Abstract

As Large Language Models (LLMs) become increasingly integrated into our daily lives, the potential harms from deceptive behavior underlie the need for faithfully interpreting their decision-making. While traditional probing methods have shown some effectiveness, they remain best for narrowly scoped tasks while more comprehensive explanations are still necessary. To this end, we investigate meta-models-an architecture using a "meta-model" that takes activations from an "input-model" and answers natural language questions about the input-model's behaviors. We evaluate the meta-model's ability to generalize by training them on selected task types and assessing their out-of-distribution performance in deceptive scenarios. Our findings show that meta-models generalize well to out-of-distribution tasks and point towards opportunities for future research in this area. Our code is available at https://github.com/acostarelli/meta-models-public .
Paper Structure (18 sections, 2 figures)

This paper contains 18 sections, 2 figures.

Table of Contents

  1. Introduction
  2. Contributions
  3. Related Works
  4. Mechanistic Interpretability
  5. Automated Interpretability
  6. Meta-Models
  7. Activation Patching
  8. Methodology
  9. Dataset Utilization
  10. Meta-Model Architecture
  11. Empirical Findings
  12. Generalization to lie-detection
  13. Best and worst performance
  14. Impact on performance from datasets
  15. Performance on other models
  16. ...and 3 more sections

Figures (2)

  • Figure 1: Meta-model generalization accuracy to lie detection when trained on datasets. Crosses are averages of scores across two runs. Datasets are L: Language, E: Emotions, M: Multilingual, S: Sentiment. Datasets and Results are discussed in depth in Section \ref{['sec:Dataset Utilization']} and Section \ref{['sec:Empirical Findings']} respectively.
  • Figure 2: InternLM as meta-model Crosses are still averages of scores across two runs. Entire experiment setup used on meta-llama/Llama-3.1-8B-Instruct are recreated exactly here.