Decoding Neural Emotion Patterns through Large Language Model Embeddings

TL;DR

The paper introduces a cost-effective framework to map natural language emotion to brain regions without neuroimaging, using 1,536-dim embeddings, PCA-based 3D mapping, and clustering to 29 brain regions. It leverages three datasets (DIAC-WOZ, GoEmotions, Schema-Guided Dialogue) to differentiate healthy vs depressed language, delineate emotion intensity hierarchies, and compare human vs LLM-generated text. Across three experiments, the approach yields neuroanatomically plausible activation patterns, with depressed language showing reduced cortical/subcortical engagement and humans showing stronger limbic/memory involvement relative to LLMs. While the mappings are computational and require neuroimaging validation, the method provides scalable, interpretable hypotheses and a public codebase for further validation and AI-emotion benchmarking.

Abstract

Understanding how emotional expression in language relates to brain function is a challenge in computational neuroscience and affective computing. Traditional neuroimaging is costly and lab-bound, but abundant digital text offers new avenues for emotion-brain mapping. Prior work has largely examined neuroimaging-based emotion localization or computational text analysis separately, with little integration. We propose a computational framework that maps textual emotional content to anatomically defined brain regions without requiring neuroimaging. Using OpenAI's text-embedding-ada-002, we generate high-dimensional semantic representations, apply dimensionality reduction and clustering to identify emotional groups, and map them to 18 brain regions linked to emotional processing. Three experiments were conducted: i) analyzing conversational data from healthy vs. depressed subjects (DIAC-WOZ dataset) to compare mapping patterns, ii) applying the method to the GoEmotions dataset and iii) comparing human-written text with large language model (LLM) responses to assess differences in inferred brain activation. Emotional intensity was scored via lexical analysis. Results showed neuroanatomically plausible mappings with high spatial specificity. Depressed subjects exhibited greater limbic engagement tied to negative affect. Discrete emotions were successfully differentiated. LLM-generated text matched humans in basic emotion distribution but lacked nuanced activation in empathy and self-referential regions (medial prefrontal and posterior cingulate cortex). This cost-effective, scalable approach enables large-scale analysis of naturalistic language, distinguishes between clinical populations, and offers a brain-based benchmark for evaluating AI emotional expression.

Figures (8)

• Figure 1: Five-step computational pipeline to convert natural language text to embeddings, reduce dimensionality, cluster to emotional groups, map to brain regions and calculate activations.
• Figure 2: Text embedding clusters mapped to brain regions via PCA dimension reduction based on neuro-scientifically plausible regions.
• Figure 3: Emotion to brain region assignment hierarchy applied in this study.
• Figure 4: Comparison of model-derived activation estimates per brain region for healthy versus depressed subjects.
• Figure 5: 3D rendering of emotion predicted activation differences (Table \ref{['tab:stats1']}) showing lateral (top) and ventral (bottom) views between healthy (left) and depressed (right) subjects. The color bar indicates normalized activation magnitudes, ranging from 0.07 (white) to 0.100 (red).