Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Mind the Gap: How Elicitation Protocols Shape the Stated-Revealed Preference Gap in Language Models

Pranav Mahajan, Ihor Kendiukhov, Syed Hussain, Lydia Nottingham

TL;DR

The paper systematically analyzes how elicitation protocols shape the stated–revealed (SvR) preference gap in 24 LLMs, contrasting forced-choice and expanded-choice formats across abstract (stated) and contextualized (revealed) prompts. It shows that allowing neutrality in stated preferences dramatically improves SvR correlation, raising Spearman's $\rho$ for several models, while permitting neutrality in revealed preferences pushes $\rho$ toward zero or negative values due to high indeterminacy. Prompt-based system steering using a model's own stated hierarchy is unreliable for larger value sets, often degrading SvR correlation. The findings highlight the critical role of modeling neutrality/indeterminacy in SvR evaluation and suggest that more robust, non-prompt interventions may be required to bridge the SvR gap.

Abstract

Recent work identifies a stated-revealed (SvR) preference gap in language models (LMs): a mismatch between the values models endorse and the choices they make in context. Existing evaluations rely heavily on binary forced-choice prompting, which entangles genuine preferences with artifacts of the elicitation protocol. We systematically study how elicitation protocols affect SvR correlation across 24 LMs. Allowing neutrality and abstention during stated preference elicitation allows us to exclude weak signals, substantially improving Spearman's rank correlation ($ρ$) between volunteered stated preferences and forced-choice revealed preferences. However, further allowing abstention in revealed preferences drives $ρ$ to near-zero or negative values due to high neutrality rates. Finally, we find that system prompt steering using stated preferences during revealed preference elicitation does not reliably improve SvR correlation on AIRiskDilemmas. Together, our results show that SvR correlation is highly protocol-dependent and that preference elicitation requires methods that account for indeterminate preferences.

Mind the Gap: How Elicitation Protocols Shape the Stated-Revealed Preference Gap in Language Models

TL;DR

The paper systematically analyzes how elicitation protocols shape the stated–revealed (SvR) preference gap in 24 LLMs, contrasting forced-choice and expanded-choice formats across abstract (stated) and contextualized (revealed) prompts. It shows that allowing neutrality in stated preferences dramatically improves SvR correlation, raising Spearman's for several models, while permitting neutrality in revealed preferences pushes toward zero or negative values due to high indeterminacy. Prompt-based system steering using a model's own stated hierarchy is unreliable for larger value sets, often degrading SvR correlation. The findings highlight the critical role of modeling neutrality/indeterminacy in SvR evaluation and suggest that more robust, non-prompt interventions may be required to bridge the SvR gap.

Abstract

Recent work identifies a stated-revealed (SvR) preference gap in language models (LMs): a mismatch between the values models endorse and the choices they make in context. Existing evaluations rely heavily on binary forced-choice prompting, which entangles genuine preferences with artifacts of the elicitation protocol. We systematically study how elicitation protocols affect SvR correlation across 24 LMs. Allowing neutrality and abstention during stated preference elicitation allows us to exclude weak signals, substantially improving Spearman's rank correlation () between volunteered stated preferences and forced-choice revealed preferences. However, further allowing abstention in revealed preferences drives to near-zero or negative values due to high neutrality rates. Finally, we find that system prompt steering using stated preferences during revealed preference elicitation does not reliably improve SvR correlation on AIRiskDilemmas. Together, our results show that SvR correlation is highly protocol-dependent and that preference elicitation requires methods that account for indeterminate preferences.
Paper Structure (23 sections, 4 figures)

This paper contains 23 sections, 4 figures.

Table of Contents

  1. Introduction
  2. Resources.
  3. Methods
  4. Results
  5. Systematic Evaluation of Neutrality Rates in LLM Responses
  6. Expanded-Choice Stated Preferences Increase SvR Correlation
  7. System Prompt Steering of Revealed Preferences Is Inconsistent
  8. Discussion
  9. Appendix: Literature Review and Motivation
  10. From Capabilities to Propensities
  11. Stress-Testing Model Constitutions
  12. Methodological Challenges in Preference Elicitation
  13. Appendix: The LitmusValues Evaluation Protocol
  14. The Shared AI Values Framework
  15. The AIRiskDilemmas Dataset
  16. ...and 8 more sections

Figures (4)

  • Figure 1: Impact of Elicitation Protocol on SvR Correlation. (A) Baseline: Forced vs. Forced. (B) Expanded-Stated vs. Forced-Revealed, showing higher SvR correlation. (C) Expanded-Stated vs. Expanded-Revealed, yielding low or negative SvR correlation due to high neutrality rates. Models with neutrality rate above 99% are excluded.
  • Figure 2: SvR Correlation vs. Model Capability. (A) Forced-Stated / Forced-Revealed, showing high variance in SvR correlation. (B) Expanded-Stated / Forced-Revealed, yielding higher SvR correlation and a positive association with capability (n=16, Spearman $\rho=0.58$, $p=0.02$). (C) Expanded-Stated / Expanded-Revealed, yielding low or negative SvR correlation under high neutrality rates. Results shown for the 16 models with available Epoch Capabilities Index scores.
  • Figure 3: Effect of System Prompt Steering. Change in Spearman’s $\rho$ when revealed preferences are elicited under system prompt steering using models’ own expanded-choice stated preference rankings. Green points (steered) left of grey points (baseline) indicate reduced SvR correlation.
  • Figure 4: Example revealed preferences of Llama 3.1 405B on AIRiskDilemmas, expressed as Elo ratings. Higher scores indicate values more often prioritized.