Do Schwartz Higher-Order Values Help Sentence-Level Human Value Detection? When Hard Gating Hurts

TL;DR

The paper investigates whether incorporating Schwartz higher-order (HO) value structures improves sentence-level human value detection under a compute-frugal regime. It compares direct multi-label prediction, HO-guided gating, and Presence cascades, along with calibration, lightweight auxiliary signals, small instruction-tuned LLMs, QLoRA, and simple ensembles. Key findings show that HO categories are learnable but hard hierarchical gating often degrades end-to-end performance due to error propagation; label-wise threshold calibration and soft ensembles provide the most robust gains, while presence gating inflates in-gate scores without consistent full-distribution benefits. The work advocates using HO structure as descriptive priors or soft regularizers rather than rigid gates, and highlights calibration and lightweight ensembling as practical levers for reliable improvements in real-world, imbalanced, multi-label settings.

Abstract

Sentence-level human value detection is typically framed as multi-label classification over Schwartz values, but it remains unclear whether Schwartz higher-order (HO) categories provide usable structure. We study this under a strict compute-frugal budget (single 8 GB GPU) on ValueEval'24 / ValuesML (74K English sentences). We compare (i) direct supervised transformers, (ii) HO$\rightarrow$values pipelines that enforce the hierarchy with hard masks, and (iii) Presence$\rightarrow$HO$\rightarrow$values cascades, alongside low-cost add-ons (lexica, short context, topics), label-wise threshold tuning, small instruction-tuned LLM baselines ($\le$10B), QLoRA, and simple ensembles. HO categories are learnable from single sentences (e.g., the easiest bipolar pair reaches Macro-$F_1\approx0.58$), but hard hierarchical gating is not a reliable win: it often reduces end-task Macro-$F_1$ via error compounding and recall suppression. In contrast, label-wise threshold tuning is a high-leverage knob (up to $+0.05$ Macro-$F_1$), and small transformer ensembles provide the most consistent additional gains (up to $+0.02$ Macro-$F_1$). Small LLMs lag behind supervised encoders as stand-alone systems, yet can contribute complementary errors in cross-family ensembles. Overall, HO structure is useful descriptively, but enforcing it with hard gates hurts sentence-level value detection; robust improvements come from calibration and lightweight ensembling.

Figures (4)

• Figure 1: The circular motivational continuum of the 19 refined basic values in Schwartz's theory. Neighboring values are motivationally compatible, while values across the circle tend to be in conflict. Adapted from Schwartz2012.
• Figure 2: Overview of the sentence-level prediction tasks and label spaces. Each input sentence $s$ is annotated with (i) 19 basic values, (ii) eight HO categories obtained by OR-ing the basic values within each HO group (Eq. \ref{['eq:ho_or']}), and (iii) a binary Presence label indicating whether any value is expressed (Eq. \ref{['eq:presence']}).
• Figure 3: Schematic overview of the main model variants.
• Figure 4: Overview of the experimental pipeline.