From Predictions to Decisions: The Importance of Joint Predictive Distributions

TL;DR

This work argues that accurate joint predictive distributions, not just marginal predictions, are crucial for effective decision-making across combinatorial, sequential, and bandit problems. By formalizing a KL-based metric $ d_{\mathrm{KL}}^\tau $ and showing its universality for downstream tasks, the authors connect predictive accuracy to actionable decisions, even when marginal accuracy appears sufficient. They introduce an approximate Thompson sampling algorithm that leverages joint predictive samples and prove a new regret bound, demonstrating near-optimal performance under suitable joint-predictive accuracy. The results offer a principled information-theoretic lens for uncertainty estimation in sequential decision-making and suggest practical pathways for designing agents that retain and exploit joint predictive structure.

Abstract

A fundamental challenge for any intelligent system is prediction: given some inputs, can you predict corresponding outcomes? Most work on supervised learning has focused on producing accurate marginal predictions for each input. However, we show that for a broad class of decision problems, accurate joint predictions are required to deliver good performance. In particular, we establish several results pertaining to combinatorial decision problems, sequential predictions, and multi-armed bandits to elucidate the essential role of joint predictive distributions. Our treatment of multi-armed bandits introduces an approximate Thompson sampling algorithm and analytic techniques that lead to a new kind of regret bound.

Theorems & Definitions (8)

• Proposition 2.1
• Theorem 4.1
• Theorem 5.1
• proof
• proof
• Lemma 2.1
• proof
• proof