Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A tutorial on learning from preferences and choices with Gaussian Processes

Alessio Benavoli, Dario Azzimonti

TL;DR

This work presents a unified Gaussian Process framework for learning from preferences and choices, covering object and label data as well as choice behavior. It introduces nine GP-based models that accommodate rational utilities, random utility perturbations, discernibility limits, multiple utilities, and two-argument representations, along with extensions to label rankings and paired comparisons. The approach relies on flexible likelihoods (probit, Thurstone, Plackett–Luce, etc.) atop GP priors, enabling uncertainty quantification and scalable inference via inducing points and approximate methods; a prefGP library is provided for practical deployment. Applications span transportation mode selection, gaming platform rankings, subjective judgments (ellipses), and recommender systems (MovieLens), illustrating both predictive accuracy and interpretability of latent utilities. The work highlights connections to preferential Bayesian optimization and outlines future directions, including mixed data sources and monotonicity/convexity constraints to further enhance model fidelity and applicability.

Abstract

Preference modelling lies at the intersection of economics, decision theory, machine learning and statistics. By understanding individuals' preferences and how they make choices, we can build products that closely match their expectations, paving the way for more efficient and personalised applications across a wide range of domains. The objective of this tutorial is to present a cohesive and comprehensive framework for preference learning with Gaussian Processes (GPs), demonstrating how to seamlessly incorporate rationality principles (from economics and decision theory) into the learning process. By suitably tailoring the likelihood function, this framework enables the construction of preference learning models that encompass random utility models, limits of discernment, and scenarios with multiple conflicting utilities for both object- and label-preference. This tutorial builds upon established research while simultaneously introducing some novel GP-based models to address specific gaps in the existing literature.

A tutorial on learning from preferences and choices with Gaussian Processes

TL;DR

This work presents a unified Gaussian Process framework for learning from preferences and choices, covering object and label data as well as choice behavior. It introduces nine GP-based models that accommodate rational utilities, random utility perturbations, discernibility limits, multiple utilities, and two-argument representations, along with extensions to label rankings and paired comparisons. The approach relies on flexible likelihoods (probit, Thurstone, Plackett–Luce, etc.) atop GP priors, enabling uncertainty quantification and scalable inference via inducing points and approximate methods; a prefGP library is provided for practical deployment. Applications span transportation mode selection, gaming platform rankings, subjective judgments (ellipses), and recommender systems (MovieLens), illustrating both predictive accuracy and interpretability of latent utilities. The work highlights connections to preferential Bayesian optimization and outlines future directions, including mixed data sources and monotonicity/convexity constraints to further enhance model fidelity and applicability.

Abstract

Preference modelling lies at the intersection of economics, decision theory, machine learning and statistics. By understanding individuals' preferences and how they make choices, we can build products that closely match their expectations, paving the way for more efficient and personalised applications across a wide range of domains. The objective of this tutorial is to present a cohesive and comprehensive framework for preference learning with Gaussian Processes (GPs), demonstrating how to seamlessly incorporate rationality principles (from economics and decision theory) into the learning process. By suitably tailoring the likelihood function, this framework enables the construction of preference learning models that encompass random utility models, limits of discernment, and scenarios with multiple conflicting utilities for both object- and label-preference. This tutorial builds upon established research while simultaneously introducing some novel GP-based models to address specific gaps in the existing literature.
Paper Structure (39 sections, 10 theorems, 113 equations, 24 figures, 6 tables)

This paper contains 39 sections, 10 theorems, 113 equations, 24 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Utility function based learning.
  3. Preference learning with two-argument functions.
  4. Objective of this tutorial
  5. Why Gaussian processes?
  6. Literature overview
  7. Tutorial organisation
  8. Preliminaries
  9. Preferences and Utilities
  10. Representation via utility function
  11. Gaussian process
  12. Learning from object preferences
  13. Learning consistent preferences
  14. Problems arising in the modelling of preference relations
  15. Accounting for the limit of discernibility
  16. ...and 24 more sections

Key Result

Proposition 2

(kreps1990course) Any strict preference is irreflexive, transitive and acyclic.

Figures (24)

  • Figure 1: Alice's utility for home temperature.
  • Figure 2: Sampling from a constrained region using lin-ess,
  • Figure 3: Left: learned utility functions, the mean is the continuous line and the region between the dotted-lines is the 95% credible region. Right: posterior marginal of the difference $u(20)-u(18)$.
  • Figure 4: Left: learned utility functions under Luce's model, the mean is the continuous line and the region between the dotted-lines is the 95% credible region. The other three figures are the posterior marginals of $u(20)-u(18)$, $u(15)-u(19)$ and, respectively, $u(13)-u(18)$.
  • Figure 5: $\Phi(\Delta_u/\sigma)$ for different values of $\sigma$.
  • ...and 19 more figures

Theorems & Definitions (32)

  • Definition 1
  • Proposition 2
  • Definition 3
  • Proposition 4
  • Proposition 5
  • Definition 6
  • Example 1: Thermal comfort
  • Example 2
  • Lemma 7
  • Remark 8
  • ...and 22 more