Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Context Tree Prior Distributions based on Node Weighting with exact Bayes Factors

Thiago Paulichen, Victor Freguglia

Abstract

Variable-length Markov chains (VLMCs) are a flexible class of higher-order Markov models that admit a natural representation as context trees. The Bayesian approach to VLMCs involves assigning prior distributions to both the tree structure and the transition probabilities, with inference performed by integrating over nuisance parameters. We propose a Bayesian framework for context trees based on a novel class of prior distributions on tree space. The approach extends recursive methods used to compute the marginal likelihood and identify a posterior mode tree, accommodating flexible weighting schemes corresponding to these prior distributions. The proposed method generalizes existing approaches, as this class includes, among others, distributions generated by branching processes. This flexibility enables the selection of priors based on desired properties. Our framework enables efficient posterior exploration and allows model comparison and hypothesis testing via Bayes factors for context trees. To show the power of our methods, we conducted a simulation study comparing different prior choices, along with procedures based on Bayes factors to select the maximal depth and perform model selection.

Context Tree Prior Distributions based on Node Weighting with exact Bayes Factors

Abstract

Variable-length Markov chains (VLMCs) are a flexible class of higher-order Markov models that admit a natural representation as context trees. The Bayesian approach to VLMCs involves assigning prior distributions to both the tree structure and the transition probabilities, with inference performed by integrating over nuisance parameters. We propose a Bayesian framework for context trees based on a novel class of prior distributions on tree space. The approach extends recursive methods used to compute the marginal likelihood and identify a posterior mode tree, accommodating flexible weighting schemes corresponding to these prior distributions. The proposed method generalizes existing approaches, as this class includes, among others, distributions generated by branching processes. This flexibility enables the selection of priors based on desired properties. Our framework enables efficient posterior exploration and allows model comparison and hypothesis testing via Bayes factors for context trees. To show the power of our methods, we conducted a simulation study comparing different prior choices, along with procedures based on Bayes factors to select the maximal depth and perform model selection.

Paper Structure

This paper contains 22 sections, 5 theorems, 42 equations, 8 figures, 6 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Context Trees
  3. Full rooted trees and properties
  4. VLMCs and Context Trees
  5. A Bayesian Framework to Context Trees
  6. Context-tree functions and properties
  7. Prior specification and Bayesian inference
  8. Simulations and Specific Applications
  9. Simulation - Priors comparison
  10. Data and setup
  11. Simulation results evaluation
  12. Results and comments
  13. Application - Maximal depth selection
  14. Application - Model selection
  15. Conclusions
  16. ...and 7 more sections

Key Result

Lemma 1

For a VLMC sample $\mathbf{z}$ compatible with the context tree $(\tau, \mathbf{p})$, and $\pi(\mathbf{p} \mid \tau)$ as in prior_eq2, we have the following:

Figures (8)

  • Figure 1: Example of full rooted tree with root node $\lambda$ and labels in $\mathcal{A} = \{0, 1\}$
  • Figure 2: Example of context tree $(\tau, \mathbf{p})$
  • Figure 3: Sum and maximum over all trees of the function $F$
  • Figure 4: Generator VLMC models $(\tau, \mathbf{p})$
  • Figure 5: Trees $\tau_1$ and $\tau_2$
  • ...and 3 more figures

Theorems & Definitions (17)

  • Definition 1
  • Definition 2
  • Definition 3
  • Lemma 1
  • Definition 4
  • Proposition 1
  • Theorem 1
  • Example 1
  • Proposition 2
  • Proposition 3
  • ...and 7 more