Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

When Is Generalized Bayes Bayesian? A Decision-Theoretic Characterization of Loss-Based Updating

Kenichiro McAlinn, Kōsaku Takanashi

TL;DR

This paper clarifies the boundary between Bayesian belief updating and loss-based updating by separating belief posteriors from decision posteriors. It proves that a loss-based posterior coincides with ordinary Bayes only when the loss equals an (up to scale) negative log-likelihood, and otherwise should be interpreted as a decision posterior, with calibration of the learning rate essential. It derives a decision-theoretic foundation for generalized Bayes via variational preferences and an entropy-based deviation cost, yielding the Gibbs form $q(\theta|x)\propto\pi(\theta)\exp\{-\eta L_n(\theta)\}$ as the optimal rule under sequential coherence and separability. The paper also shows that generalized marginal likelihood is not canonical evidence for decision posteriors and proposes shift-invariant predictive scoring as a constructive alternative for model comparison. Collectively, it provides a practical coherence framework (the coherence book) and actionable design steps for constructing and diagnosing loss-based updates, highlighting the central role of learning-rate calibration in non-log-loss settings.

Abstract

Loss-based updating, including generalized Bayes, Gibbs, and quasi-posteriors, replaces likelihoods by a user-chosen loss and produces a posterior-like distribution via exponential tilt. We give a decision-theoretic characterization that separates \emph{belief posteriors} -- conditional beliefs justified by the foundations of Savage and Anscombe-Aumann under a joint probability mode l-- from \emph{decision posteriors} -- randomized decision rules justified by preferences over decision rules. We make explicit that a loss-based posterior coincides with ordinary Bayes if and only if the loss is, up to scale and a data-only term, negative log-likelihood. We then show that generalized marginal likelihood is not evidence for decision posteriors, and Bayes factors are not well-defined without additional structure. In the decision posterior regime, non-degenerate posteriors require nonlinear preferences over decision rules. Under sequential coherence and separability, these lead to an entropy-penalized variational representation yielding generalized Bayes as the optimal rule.

When Is Generalized Bayes Bayesian? A Decision-Theoretic Characterization of Loss-Based Updating

TL;DR

This paper clarifies the boundary between Bayesian belief updating and loss-based updating by separating belief posteriors from decision posteriors. It proves that a loss-based posterior coincides with ordinary Bayes only when the loss equals an (up to scale) negative log-likelihood, and otherwise should be interpreted as a decision posterior, with calibration of the learning rate essential. It derives a decision-theoretic foundation for generalized Bayes via variational preferences and an entropy-based deviation cost, yielding the Gibbs form as the optimal rule under sequential coherence and separability. The paper also shows that generalized marginal likelihood is not canonical evidence for decision posteriors and proposes shift-invariant predictive scoring as a constructive alternative for model comparison. Collectively, it provides a practical coherence framework (the coherence book) and actionable design steps for constructing and diagnosing loss-based updates, highlighting the central role of learning-rate calibration in non-log-loss settings.

Abstract

Loss-based updating, including generalized Bayes, Gibbs, and quasi-posteriors, replaces likelihoods by a user-chosen loss and produces a posterior-like distribution via exponential tilt. We give a decision-theoretic characterization that separates \emph{belief posteriors} -- conditional beliefs justified by the foundations of Savage and Anscombe-Aumann under a joint probability mode l-- from \emph{decision posteriors} -- randomized decision rules justified by preferences over decision rules. We make explicit that a loss-based posterior coincides with ordinary Bayes if and only if the loss is, up to scale and a data-only term, negative log-likelihood. We then show that generalized marginal likelihood is not evidence for decision posteriors, and Bayes factors are not well-defined without additional structure. In the decision posterior regime, non-degenerate posteriors require nonlinear preferences over decision rules. Under sequential coherence and separability, these lead to an entropy-penalized variational representation yielding generalized Bayes as the optimal rule.
Paper Structure (44 sections, 11 theorems, 32 equations, 1 table)

This paper contains 44 sections, 11 theorems, 32 equations, 1 table.

Table of Contents

  1. Introduction
  2. Contributions.
  3. Outline.
  4. Belief posteriors and decision posteriors
  5. Belief posteriors: conditional beliefs from preferences over uncertain acts
  6. Decision posteriors: post-data randomized decision rules
  7. Loss-based updating as an operator and sequential coherence
  8. Standing assumptions and scope
  9. When is generalized Bayes Bayesian?
  10. Generalized Bayes coincides with Bayes if and only if the loss is log-likelihood
  11. From preferences over decision rules to generalized Bayes
  12. Axioms (variational decision posteriors).
  13. A design recipe for loss-based updating
  14. Step 1: Specify the target and loss.
  15. Step 2: Specify the composition structure
  16. ...and 29 more sections

Key Result

Theorem 1

Fix $\eta>0$, a baseline distribution $\pi$ on $\Theta$, and a measurable loss $\ell(\theta;x)$. The following are equivalent:

Theorems & Definitions (21)

  • Theorem 1: Bayesian equivalence
  • Example 1: Criterion-function loss breaks belief semantics
  • Corollary 1: $x$-normalization diagnostic for Bayesianity
  • Proposition 1: Separability implies product additivity
  • Proposition 2: KL is the unique $f$-divergence with product additivity
  • Corollary 2: Generalized Bayes as the unique optimal decision posterior
  • Proposition 3: Normalizing constants are not identified by a decision posterior
  • proof
  • Example 2: Empirical likelihood/exponential tilting conventions leave $q$ unchanged but rescale $Z$.
  • Corollary 3: Generalized Bayes factors are not well-defined evidence
  • ...and 11 more