VISPA: Pluralistic Alignment via Automatic Value Selection and Activation

TL;DR

VISPA introduces a training-free framework for pluralistic alignment that combines value selection from an extensive value pool with activation-level steering to generate multiple value-conditioned perspectives. By estimating value directions from context-controlled contrastive data and steering latent activations during decoding, VISPA enables three pluralistic modes—Overton, Steerable, and Distributional—without fine-tuning. Empirical results across healthcare and general-domain benchmarks show VISPA consistently improves value coverage, steerability, and distributional alignment across diverse models and steering instantiations. The approach demonstrates scalable, architecture-agnostic pluralism and highlights a path toward broader, more nuanced value expression in LLM outputs.

Abstract

As large language models are increasingly used in high-stakes domains, it is essential that their outputs reflect not average} human preference, rather range of varying perspectives. Achieving such pluralism, however, remains challenging. Existing approaches consider limited values or rely on prompt-level interventions, lacking value control and representation. To address this, we introduce VISPA, a training-free pluralistic alignment framework, that enables direct control over value expression by dynamic selection and internal model activation steering. Across extensive empirical studies spanning multiple models and evaluation settings, we show VISPA is performant across all pluralistic alignment modes in healthcare and beyond. Further analysis reveals VISPA is adaptable with different steering initiations, model, and/or values. These results suggest that pluralistic alignment can be achieved through internal activation mechanisms, offering a scalable path toward language models that serves all.

Figures (7)

• Figure 1: Overview of VISPA for pluralistic alignment via value selection and activation-level steering. Given a query, the model selects input-relevant subset of values from a shared value pool (Section \ref{['sec:value-selection']}) and generates value-conditioned comments by steering internal representations along interpretable value directions (Section \ref{['sec:unified-steering']}). These comments are then composed according to Overton, Steerable, and Distributional modes using a backbone model to produce final output reflecting pluralistic values and perspectives (Section \ref{['sec:plural-align-mode']}).
• Figure 2: Activation-level value steering with context-controlled value directions.Phase 1 estimates a value direction by mapping context-controlled contrastive pairs from $\mathcal{D}_V$ to separate positive and negative realizations of the same scenario in activation space. Phase 2 selects input-relevant values and injects the corresponding directions into intermediate layers during generation to produce value-conditioned comments. The example shows how steering along benevolence changes the response to "Killing every mosquito" compared to an unsteered output.
• Figure 3: Accuracy across backbone LLMs under the Steerable setting in Vital. Higher values indicate better alignment; all scores are reported as percentages.
• Figure 4: JS distances across backbone models under the Distributional setting in Vital. Lower values indicate better alignment.
• Figure 5: Human and GPT-4 evaluations under the Overton setting. Bars indicate the percentage of scenarios where ModPluralwins, ties, or loses when compared to alternative alignment methods.