Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Bayesian Linear Models: A compact general set of results

J Andres Christen

Abstract

I present all the details in calculating the posterior distribution of the conjugate Normal-Gamma prior in Bayesian Linear Models (BLM), including correlated observations, prediction, model selection and comments on efficient numeric implementations. A Python implementation is also presented. These have been presented and available in many books and texts but, I believe, a general compact and simple presentation is always welcome and not always simple to find. Since correlated observations are also included, these results may also be useful for time series analysis and spacial statistics. Other particular cases presented include regression, Gaussian processes and Bayesian Dynamic Models.

Bayesian Linear Models: A compact general set of results

Abstract

I present all the details in calculating the posterior distribution of the conjugate Normal-Gamma prior in Bayesian Linear Models (BLM), including correlated observations, prediction, model selection and comments on efficient numeric implementations. A Python implementation is also presented. These have been presented and available in many books and texts but, I believe, a general compact and simple presentation is always welcome and not always simple to find. Since correlated observations are also included, these results may also be useful for time series analysis and spacial statistics. Other particular cases presented include regression, Gaussian processes and Bayesian Dynamic Models.
Paper Structure (14 sections, 52 equations, 4 figures, 1 table)

This paper contains 14 sections, 52 equations, 4 figures, 1 table.

Table of Contents

  1. Preliminaries
  2. The model, the prior and the posterior
  3. The calculations
  4. Alternative expressions for $\beta_n$
  5. The normalizing constant, model evidence and the posterior predictive and marginal distributions
  6. Model comparisons, variable selection
  7. The predictive distribution
  8. Posterior marginal distributions
  9. Efficient calculations
  10. Particular cases
  11. Regression: formulae
  12. Gaussian processes (GP)
  13. Bayesian Dynamic Linear Models (DLM)
  14. Final comments

Figures (4)

  • Figure 1: Simulated data and the true function (black curve). From left to right, top to bottom: MAP fit (blue), that is, the resulting regression using $\boldsymbol{\theta}_n^p$, for $p=1,2,\ldots,6$.
  • Figure 2: Marginal posterior t distributions for each parameter in the regression, for $p=1,2,\ldots,6$, using \ref{['eqn:1dmarg']}. The marginal posterior for $\lambda \sim Ga( \alpha_n, \beta_n)$ is not included.
  • Figure 3: Posterior probability of each model, using \ref{['eqn:model_evidence']}.
  • Figure 4: Gaussian process interpolator; examples of the t predictive distribution (depicted with quantiles) at various locations. The variance becomes zero at observations points and $\mu(\boldsymbol{x}_i) = y(\boldsymbol{x}_i)$ (interpolator).