Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Uncertainty Quantification Via the Posterior Predictive Variance

Sanjay Chaudhuri, Dean Dustin, Bertrand Clarke

Abstract

We use the law of total variance to generate multiple expansions for the posterior predictive variance. These expansions are sums of terms involving conditional expectations and conditional variances and provide a quantification of the sources of predictive uncertainty. Since the posterior predictive variance is fixed given the model, it represents a constant quantity that is conserved over these expansions. The terms in the expansions can be assessed in absolute or relative sense to understand the main contributors to the length of prediction intervals. We quantify the term-wise uncertainty across expansions varying in the number of terms and the order of conditionates. In particular, given that a specific term in one expansion is small or zero, we identify the other terms in other expansions that must also be small or zero. We illustrate this approach to predictive model assessment in several well-known models.

Uncertainty Quantification Via the Posterior Predictive Variance

Abstract

We use the law of total variance to generate multiple expansions for the posterior predictive variance. These expansions are sums of terms involving conditional expectations and conditional variances and provide a quantification of the sources of predictive uncertainty. Since the posterior predictive variance is fixed given the model, it represents a constant quantity that is conserved over these expansions. The terms in the expansions can be assessed in absolute or relative sense to understand the main contributors to the length of prediction intervals. We quantify the term-wise uncertainty across expansions varying in the number of terms and the order of conditionates. In particular, given that a specific term in one expansion is small or zero, we identify the other terms in other expansions that must also be small or zero. We illustrate this approach to predictive model assessment in several well-known models.
Paper Structure (17 sections, 8 theorems, 150 equations, 5 figures)

This paper contains 17 sections, 8 theorems, 150 equations, 5 figures.

Table of Contents

  1. The Setting and Intuition
  2. Two-term and Three-term Expansions of Posterior Predictive Variance
  3. Two-term Expansions
  4. Three-term Expansions
  5. Uncertainty Quantification for Two and Three-term Expansions
  6. Uncertainty Quantification under Structural Assumptions
  7. Uncertainty Quantification under posterior independence
  8. General Expansions of the PPV
  9. Uncertainty Quantification under Structural Conditions
  10. Uncertainty Quantification under Posterior Independence
  11. More General Expansions of the PPV
  12. Term-wise Relative Size of Uncertainty
  13. Two-Way Random Coefficient Model
  14. Application to the Challenger disaster data
  15. Discussion
  16. ...and 2 more sections

Key Result

Theorem 3.1

Suppose Assumption asmp:condInd holds. Then, the term eq:1b in the three-term expansion LTV2termgen is zero. Furthermore, the expansion LTV2termgen reduces to a two-term expansion conditional on $V_1$ and $\cal{D}$.

Figures (5)

  • Figure 1: Schematic diagram of a Hidden Markov model. With the definitions of $V_1$, $V_2$ and $\mathcal{D}$ as indicated, the conditional independence relationships $V_1 \perp\!\!\!\perp V_2\mid \mathcal{D}$ and $Y_{n+1} \perp\!\!\!\perp V_2\mid (V_2,\mathcal{D})$ hold.
  • Figure 2: Diagram for a 'vertical' three-level HM. It is easy to think of $V_1$ is a parameter and $V_2$ as a hyperparameter. Further conditioning, e.g., by including a $V_3$ would extend the diagram upwards.
  • Figure 3: In the implication scheme above, the two-sided implications shown by $\Longleftrightarrow$ (e.g. $T^{\{1,2,3\}}_3=0~\Longleftrightarrow~T^{\{2,1,3\}}_3=0$, or $T^{\{1,2,3\}}_1=0~\Longleftrightarrow~T^{\{1,3,2\}}_1=0$) follow from Clause 1. of Theorem \ref{['thm:genExp']} without any extra assumptions. The one-sided implications (e.g. $T^{\{1,2,3\}}_3=0~\Longrightarrow~T^{\{1,3,2\}}_2=0~\Longrightarrow~T^{\{3,1,2\}}_1=0$) follow from Clause 2 of the same theorem, and require specific posterior independence conditions to hold. In particular, those implications that require the relation $V_3 \perp\!\!\!\perp (V_1, V_2)\mid \mathcal{D}$ to hold are denoted by plain right arrows. On the other hand, those require $V_2 \perp\!\!\!\perp (V_1,V_3)\mid \mathcal{D}$ are denoted by dashed right arrows. Finally the implications that hold under $V_1 \perp\!\!\!\perp (V_2,V_3)\mid \mathcal{D}$ are shown by dotted right arrows.
  • Figure 4: The value and the proportion of the three terms in \ref{['2wayANOVAvardecomp0']}. Figures \ref{['fig:termT']} and \ref{['fig:propT']} are for $\sigma^2_{\tau}=\sigma^2_{\beta}=2$, $\sigma^2_{\epsilon}=1$, $B=2$, and varying $T$. For Figures \ref{['fig:termB']} and \ref{['fig:propB']}, we have set $B=2$, $T=3$, $\sigma^2_{\tau}=5$, $\sigma^2_{\epsilon}=1$, and let $\sigma^2_{\beta}$ vary.
  • Figure 5: Histogram of the posterior predicted failure probabilities $p_{t=31,s=200}$ for the Challenger space shuttle on the day of its launch.

Theorems & Definitions (31)

  • Example 2.1
  • Example 2.2
  • Example 2.3
  • Example 2.4
  • Example 2.5
  • Example 2.6
  • Example 2.7
  • Example 2.8
  • Theorem 3.1
  • proof
  • ...and 21 more