Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Are Emotions Arranged in a Circle? Geometric Analysis of Emotion Representations via Hyperspherical Contrastive Learning

Yusuke Yamauchi, Akiko Aizawa

TL;DR

The paper tackles whether emotions can be represented as a circle in language-model embeddings and tests this by inducing circular embeddings on a hypersphere using nGPT with three contrastive losses (SINCERE, SoftCSE, CircularCSE) guided by an Empirical Circumplex Model (ECM). It demonstrates that CircularCSE improves interpretability and robustness to dimensionality reduction but sacrifices discriminative performance in high-dimensional or fine-grained settings, while SINCERE offers stronger separation at the cost of circular alignment. The work highlights a fundamental trade-off between human-interpretable geometry and conventional DL discriminability, informing when to prioritize manifold structure versus classification accuracy. Overall, the findings provide a principled view on integrating psychological models into deep learning and motivate future work on multi-dimensional or multimodal emotion representations.

Abstract

Psychological research has long utilized circumplex models to structure emotions, placing similar emotions adjacently and opposing ones diagonally. Although frequently used to interpret deep learning representations, these models are rarely directly incorporated into the representation learning of language models, leaving their geometric validity unexplored. This paper proposes a method to induce circular emotion representations within language model embeddings via contrastive learning on a hypersphere. We show that while this circular alignment offers superior interpretability and robustness against dimensionality reduction, it underperforms compared to conventional designs in high-dimensional settings and fine-grained classification. Our findings elucidate the trade-offs involved in applying psychological circumplex models to deep learning architectures.

Are Emotions Arranged in a Circle? Geometric Analysis of Emotion Representations via Hyperspherical Contrastive Learning

TL;DR

The paper tackles whether emotions can be represented as a circle in language-model embeddings and tests this by inducing circular embeddings on a hypersphere using nGPT with three contrastive losses (SINCERE, SoftCSE, CircularCSE) guided by an Empirical Circumplex Model (ECM). It demonstrates that CircularCSE improves interpretability and robustness to dimensionality reduction but sacrifices discriminative performance in high-dimensional or fine-grained settings, while SINCERE offers stronger separation at the cost of circular alignment. The work highlights a fundamental trade-off between human-interpretable geometry and conventional DL discriminability, informing when to prioritize manifold structure versus classification accuracy. Overall, the findings provide a principled view on integrating psychological models into deep learning and motivate future work on multi-dimensional or multimodal emotion representations.

Abstract

Psychological research has long utilized circumplex models to structure emotions, placing similar emotions adjacently and opposing ones diagonally. Although frequently used to interpret deep learning representations, these models are rarely directly incorporated into the representation learning of language models, leaving their geometric validity unexplored. This paper proposes a method to induce circular emotion representations within language model embeddings via contrastive learning on a hypersphere. We show that while this circular alignment offers superior interpretability and robustness against dimensionality reduction, it underperforms compared to conventional designs in high-dimensional settings and fine-grained classification. Our findings elucidate the trade-offs involved in applying psychological circumplex models to deep learning architectures.
Paper Structure (27 sections, 30 equations, 14 figures, 4 tables)

This paper contains 27 sections, 30 equations, 14 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Overview
  5. Preliminary
  6. Head Architecture
  7. Training Objective
  8. Experiments
  9. Experimental Setup
  10. Evaluation Metrics
  11. Results
  12. Analysis
  13. Conflict between Psychological and Deep Learning Models
  14. Effects of Spherical Constraints
  15. Discussion and Conclusion
  16. ...and 12 more sections

Figures (14)

  • Figure 1: (a) An example of the psychological circumplex model of emotion. (b)(c)(d) PCA plots of the embeddings from the models trained in this study.
  • Figure 2: Overview of our experimental framework. (Left) Dataset construction procedure. Corresponding emotion labels are extracted or synthesized to reproduce the circumplex emotion structure. (Right) Training of GPT or nGPT heads across three backbone architectures using three distinct loss functions.
  • Figure 3: Our empirical circumplex model (ECM) and definition of Circumplex Distance (CD).
  • Figure 4: Average cosine similarity between emotion label pairs of mE5
  • Figure 5: Visualization of embedding representations from each module of the GPT and nGPT heads using Multidimensional Scaling (MDS). Additional results for other models are presented in Appendix \ref{['sec: module visualization']}.
  • ...and 9 more figures