Human Values in a Single Sentence: Moral Presence, Hierarchies, and Transformer Ensembles on the Schwartz Continuum

TL;DR

This work investigates sentence-level detection of Schwartz's 19 basic values in the ValueEval'24 English data, formulating two tasks: moral presence and multi-label value prediction. It compares direct DeBERTa-based multi-label classifiers, presence-gated hierarchies, instruction-tuned LLMs, and small ensembles under an 8 GB GPU budget, finding that presence can be detected with ~$F_1$ of about 0.74, while gating provides limited gains. Lightweight signals (short-range context, LIWC/eMFD/MJD features, topics) and small ensembles yield modest but robust improvements, with a soft-voting DeBERTa ensemble achieving the best macro-$F_1$ (~0.332), outperforming LLM baselines. The study concludes that moderately sized supervised encoders with careful calibration and ensembling offer a strong, compute-efficient baseline for fine-grained value detection, and suggests future work on richer value structure and sentence-in-document context to boost performance.

Abstract

We study sentence-level identification of the 19 values in the Schwartz motivational continuum as a concrete formulation of human value detection in text. The setting - out-of-context sentences from news and political manifestos - features sparse moral cues and severe class imbalance. This combination makes fine-grained sentence-level value detection intrinsically difficult, even for strong modern neural models. We first operationalize a binary moral presence task ("does any value appear?") and show that it is learnable from single sentences (positive-class F1 $\approx$ 0.74 with calibrated thresholds). We then compare a presence-gated hierarchy to a direct multi-label classifier under matched compute, both based on DeBERTa-base and augmented with lightweight signals (prior-sentence context, LIWC-22/eMFD/MJD lexica, and topic features). The hierarchy does not outperform direct prediction, indicating that gate recall limits downstream gains. We also benchmark instruction-tuned LLMs - Gemma 2 9B, Llama 3.1 8B, Mistral 8B, and Qwen 2.5 7B - in zero-/few-shot and QLoRA setups and build simple ensembles; a soft-vote supervised ensemble reaches macro-F1 0.332, significantly surpassing the best single supervised model and exceeding prior English-only baselines. Overall, in this scenario, lightweight signals and small ensembles yield the most reliable improvements, while hierarchical gating offers limited benefit. We argue that, under an 8 GB single-GPU constraint and at the 7-9B scale, carefully tuned supervised encoders remain a strong and compute-efficient baseline for structured human value detection, and we outline how richer value structure and sentence-in-document context could further improve performance.

Figures (4)

• Figure 1: Circular motivational continuum of the 19 refined basic values in Schwartz's theory. Neighbouring values are motivationally compatible, whereas values on opposite sides of the circle tend to be in conflict. Adapted from Schwartz2012.
• Figure 2: Sentence-level label space and prediction tasks.
• Figure 3: Overview of the model families considered in this work.
• Figure 4: Overview of the selection and evaluation of models: threshold tuning, champion selection, and test evaluation.