Is Sentiment Banana-Shaped? Exploring the Geometry and Portability of Sentiment Concept Vectors

TL;DR

This paper evaluates Concept Vector Projection (CVP), a projection-based method that treats sentiment as a linear direction in embedding space to produce continuous, multilingual sentiment scores. By testing CVP across Emobank, Facebook, and Fiction4 (English and Danish), it demonstrates strong cross-domain portability while also examining the method's linearity through a banana-shaped sentiment manifold. CVP generalizes beyond valence to arousal and dominance, albeit with reduced accuracy, and reveals that neutrality encodes additional semantic structure, challenging a strict linear view. The findings suggest CVP captures generalizable sentiment patterns suitable for humanities research, with future work needed to incorporate non-linear structure and broader linguistic coverage.

Abstract

Use cases of sentiment analysis in the humanities often require contextualized, continuous scores. Concept Vector Projections (CVP) offer a recent solution: by modeling sentiment as a direction in embedding space, they produce continuous, multilingual scores that align closely with human judgments. Yet the method's portability across domains and underlying assumptions remain underexplored. We evaluate CVP across genres, historical periods, languages, and affective dimensions, finding that concept vectors trained on one corpus transfer well to others with minimal performance loss. To understand the patterns of generalization, we further examine the linearity assumption underlying CVP. Our findings suggest that while CVP is a portable approach that effectively captures generalizable patterns, its linearity assumption is approximate, pointing to potential for further development.

Figures (8)

• Figure 1: A visualization of how the Concept Vector Projection is constructed. It shows how to construct a positive-negative concept vector to predict sentiment in an unlabeled corpus in a continuous space.
• Figure 2: Scatterplot visualizing the Fiction4 embeddings projected onto the Fiction4 Pos-Neg Sentiment vector and the corresponding Neutral-Component. The Margin plots are Kernel Density Estimations of the label distributions. All dimensions are Z-score normalized to make the projections interpretable.
• Figure 3: Cosine similarity between Concept Vectors for each corpus (values in each cell). Internal correlations among neg-pos, neut-pos, and neg-neut pairs are strong, with neut-pos and neg-neut closer to neg-pos, reflecting a centrality of the negative–positive axis across corpora.
• Figure 4: Correlation (Spearman's $\rho$) between transformer model (xlm-R-b) and human scores in the Facebook dataset.
• Figure 5: Relation between Concept Vector Projection scores (y-axis) and human scores (x-axis) on standardized valence across datasets. On top of each figure, the training set (on the left of the arrow) and the test set (on the right of the arrow) are shown.