Taking a SEAT: Predicting Value Interpretations from Sentiment, Emotion, Argument, and Topic Annotations

TL;DR

This paper tackles the subjectivity of human value interpretations and investigates whether a large language model can predict an individual's value interpretations by conditioning on multi-dimensional SEAT annotations (Sentiment, Emotion, Argument, Topic). By framing value prediction as a multi-label task and testing zero-shot and few-shot prompting on an energy-transition dataset annotated by five individuals, the authors show that providing all SEAT dimensions yields the strongest predictions, with micro $F_1$-scores improving over baselines (up to ~0.44 on the best settings). The work highlights that single SEAT dimensions offer limited gains, while an integrated, contextual SEAT signal captures the complexity of value interpretation, suggesting a path toward personalized AI that respects diverse perspectives. However, the study remains small in scale, with notable inter-individual differences and generalization challenges, motivating future work across more annotators, domains, and multi-modal cues to validate and extend these findings.

Abstract

Our interpretation of value concepts is shaped by our sociocultural background and lived experiences, and is thus subjective. Recognizing individual value interpretations is important for developing AI systems that can align with diverse human perspectives and avoid bias toward majority viewpoints. To this end, we investigate whether a language model can predict individual value interpretations by leveraging multi-dimensional subjective annotations as a proxy for their interpretive lens. That is, we evaluate whether providing examples of how an individual annotates Sentiment, Emotion, Argument, and Topics (SEAT dimensions) helps a language model in predicting their value interpretations. Our experiment across different zero- and few-shot settings demonstrates that providing all SEAT dimensions simultaneously yields superior performance compared to individual dimensions and a baseline where no information about the individual is provided. Furthermore, individual variations across annotators highlight the importance of accounting for the incorporation of individual subjective annotators. To the best of our knowledge, this controlled setting, although small in size, is the first attempt to go beyond demographics and investigate the impact of annotation behavior on value prediction, providing a solid foundation for future large-scale validation.

Figures (4)

• Figure 1: $F_1$-score resulting from including different auxiliary information in the prompt, averaged over the five annotators. The horizontal dashed line displays the ZS baseline ($F_1=0.217$) averaged over the five annotators.
• Figure 2: $F_1$-score by annotator, averaged over the different methods used to provide auxiliary information (i.e., the four rows in Table \ref{['table:settings']}).
• Figure 3: $F_1$-score by annotator, averaged over the different provided auxiliary information (i.e., the five columns in Table \ref{['table:settings']}).
• Figure 4: Predicted label change compared to the zero-shot ZS baseline, measured as the symmetric difference $(|A \Delta B| / |A|)$ percentage, averaged over the five annotators.