Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Optimal Use of Preferences in Artificial Intelligence Algorithms

Joshua S. Gans

TL;DR

This paper provides decision problem agnostic conditions under which separation training preference free and applying preferences ex post is optimal, and provides design guidance: preserve optionality through post-processing when objectives may shift; embed preferences when decision-stage frictions dominate.

Abstract

Machine learning systems embed preferences either in training losses or through post-processing of calibrated predictions. Applying information design methods from Strack and Yang (2024), this paper provides decision problem agnostic conditions under which separation training preference free and applying preferences ex post is optimal. Unlike prior work that requires specifying downstream objectives, the welfare results here apply uniformly across decision problems. The key primitive is a diminishing-value-of-information condition: relative to a fixed (normalised) preference-free loss, preference embedding makes informativeness less valuable at the margin, inducing a mean-preserving contraction of learned posteriors. Because the value of information is convex in beliefs, preference-free training weakly dominates for any expected utility decision problem. This provides theoretical foundations for modular AI pipelines that learn calibrated probabilities and implement asymmetric costs through downstream decision rules. However, separation requires users to implement optimal decision rules. When cognitive constraints bind, as documented in human AI decision-making, preference embedding can dominate by automating threshold computation. These results provide design guidance: preserve optionality through post-processing when objectives may shift; embed preferences when decision-stage frictions dominate.

Optimal Use of Preferences in Artificial Intelligence Algorithms

TL;DR

This paper provides decision problem agnostic conditions under which separation training preference free and applying preferences ex post is optimal, and provides design guidance: preserve optionality through post-processing when objectives may shift; embed preferences when decision-stage frictions dominate.

Abstract

Machine learning systems embed preferences either in training losses or through post-processing of calibrated predictions. Applying information design methods from Strack and Yang (2024), this paper provides decision problem agnostic conditions under which separation training preference free and applying preferences ex post is optimal. Unlike prior work that requires specifying downstream objectives, the welfare results here apply uniformly across decision problems. The key primitive is a diminishing-value-of-information condition: relative to a fixed (normalised) preference-free loss, preference embedding makes informativeness less valuable at the margin, inducing a mean-preserving contraction of learned posteriors. Because the value of information is convex in beliefs, preference-free training weakly dominates for any expected utility decision problem. This provides theoretical foundations for modular AI pipelines that learn calibrated probabilities and implement asymmetric costs through downstream decision rules. However, separation requires users to implement optimal decision rules. When cognitive constraints bind, as documented in human AI decision-making, preference embedding can dominate by automating threshold computation. These results provide design guidance: preserve optionality through post-processing when objectives may shift; embed preferences when decision-stage frictions dominate.
Paper Structure (48 sections, 23 theorems, 106 equations, 2 figures, 1 table)

This paper contains 48 sections, 23 theorems, 106 equations, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. Model Setup and Preliminary Results
  3. Loss Functions and Post-Processing
  4. Training and Learning Frictions
  5. Existence.
  6. Learning Friction.
  7. Key Condition.
  8. Main Results: Contraction and Separation
  9. Preference embedding reduces learned informativeness
  10. Separation principle: learn probabilities, then decide
  11. Closed-form illustration
  12. A common decision environment
  13. Loss Choice and Implementation Under Separation
  14. Design class and the meaning of "optimal among variants"
  15. Choosing a preference-free baseline
  16. ...and 33 more sections

Key Result

Lemma 1

For any decision problem $(\mathcal{A},u)$, the function $V:[0,1]\to\mathbb{R}$ is convex.

Figures (2)

  • Figure 1: Illustration of $Q \succeq_{\mathrm{cx}} Q'$: $Q'$ is a mean-preserving contraction of $Q$ (schematic).
  • Figure 2: Two locations for preferences in a prediction pipeline: embed them in training (A) or apply them after learning calibrated probabilities (B).

Theorems & Definitions (51)

  • Lemma 1: Convexity of indirect value
  • proof
  • Definition 1: Bayes-plausible posterior distributions
  • Definition 2: Convex order
  • Lemma 2: More informative posteriors improve expected value
  • proof
  • Definition 3: Strictly proper scoring rule
  • Definition 4: Bayes risk
  • Proposition 1: Bayes risk and curvature for class-weighted cross-entropy
  • proof
  • ...and 41 more