Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Factored Conditional Filtering: Tracking States and Estimating Parameters in High-Dimensional Spaces

Dawei Chen, Samuel Yang-Zhao, John Lloyd, Kee Siong Ng

TL;DR

This work tackles tracking high-dimensional epidemic states while estimating associated parameters by introducing factored conditional filters. By decomposing the state space into low-dimensional clusters and conditioning state updates on a parameter space, the approach enables scalable tracking and learning in large contact networks. The authors present three algorithmic variants—factored conditional filters, including particle and variational versions—that integrate with nonconditional parameter filters. Through epidemic experiments on real-world networks, they demonstrate accurate state tracking and parameter estimation, illustrating practical impact for large-scale networked systems. The framework also lays groundwork for further theoretical analysis and partition-optimization methods in high-dimensional filtering.

Abstract

This paper introduces factored conditional filters, new filtering algorithms for simultaneously tracking states and estimating parameters in high-dimensional state spaces. The conditional nature of the algorithms is used to estimate parameters and the factored nature is used to decompose the state space into low-dimensional subspaces in such a way that filtering on these subspaces gives distributions whose product is a good approximation to the distribution on the entire state space. The conditions for successful application of the algorithms are that observations be available at the subspace level and that the transition model can be factored into local transition models that are approximately confined to the subspaces; these conditions are widely satisfied in computer science, engineering, and geophysical filtering applications. We give experimental results on tracking epidemics and estimating parameters in large contact networks that show the effectiveness of our approach.

Factored Conditional Filtering: Tracking States and Estimating Parameters in High-Dimensional Spaces

TL;DR

This work tackles tracking high-dimensional epidemic states while estimating associated parameters by introducing factored conditional filters. By decomposing the state space into low-dimensional clusters and conditioning state updates on a parameter space, the approach enables scalable tracking and learning in large contact networks. The authors present three algorithmic variants—factored conditional filters, including particle and variational versions—that integrate with nonconditional parameter filters. Through epidemic experiments on real-world networks, they demonstrate accurate state tracking and parameter estimation, illustrating practical impact for large-scale networked systems. The framework also lays groundwork for further theoretical analysis and partition-optimization methods in high-dimensional filtering.

Abstract

This paper introduces factored conditional filters, new filtering algorithms for simultaneously tracking states and estimating parameters in high-dimensional state spaces. The conditional nature of the algorithms is used to estimate parameters and the factored nature is used to decompose the state space into low-dimensional subspaces in such a way that filtering on these subspaces gives distributions whose product is a good approximation to the distribution on the entire state space. The conditions for successful application of the algorithms are that observations be available at the subspace level and that the transition model can be factored into local transition models that are approximately confined to the subspaces; these conditions are widely satisfied in computer science, engineering, and geophysical filtering applications. We give experimental results on tracking epidemics and estimating parameters in large contact networks that show the effectiveness of our approach.
Paper Structure (34 sections, 136 equations, 14 figures, 2 tables, 12 algorithms)

This paper contains 34 sections, 136 equations, 14 figures, 2 tables, 12 algorithms.

Table of Contents

  1. Introduction
  2. A General Setting for Filtering
  3. Notation
  4. The Setting
  5. Standard Filtering
  6. Conditional Filtering
  7. Factored Filtering
  8. Factored Conditional Filtering
  9. Interface between conditional and nonconditional filters
  10. Observation model synthesis
  11. Partial Definitions of Conditional Empirical Beliefs
  12. Computing $\nu_n(h_n) \otimes \lambda x. \mu_n(h_n, x)$ and $\nu_n(h_n) \odot \lambda x. \mu_n(h_n, x)$
  13. Modelling Epidemic Processes on Contact Networks
  14. Modelling States
  15. Nodes Labelled by Compartments
  16. ...and 19 more sections

Figures (14)

  • Figure 1: Compartmental models for epidemics. Left: SIS model with contact rate $\beta$ and recovery rate $\gamma$; Right: SEIRS model with contact rate $\beta$, latency rate $\sigma$, recovery rate $\gamma$, and loss of immunity rate $\rho$.
  • Figure 2: Population properties of one run of the simulation on two contact networks, Facebook and Youtube. The first row is for Facebook and the second row is for Youtube. The left column uses the parameters $\beta=0.2, \, \sigma=1/3, \, \gamma=1/14, \, \rho=1/180$; the right column uses the parameters $\beta=0.27, \, \sigma=1/2, \, \gamma=1/7, \, \rho=1/90$.
  • Figure 3: State errors of an SEIRS epidemic model for three contact networks, Gowalla, Youtube, and AS-733, using Algorithms \ref{['algo:fcf']} and \ref{['algo:spf']} for 100 independent runs in which the disease does not die out in 600 time steps for the Gowalla and the Youtube contact networks, and 733 time steps for the AS-733 dynamic contact network. The first row is for Gowalla, the second row is for Youtube, and the third row is for AS-733. The left column uses the parameters $\beta=0.2, \, \sigma=1/3, \, \gamma=1/14, \,\rho=1/180, \, \lambda_{FP} = 0.1, \,\lambda_{FN} = 0.1$; the right column uses the parameters $\beta=0.27, \, \sigma=1/2, \, \gamma=1/7, \, \rho=1/90, \, \lambda_{FP} = 0.1, \,\lambda_{FN} = 0.3$. Light gray: state errors from individual runs. Dark solid: mean state error averaged over 100 independent runs.
  • Figure 4: Parameter estimation of an SEIRS epidemic model for the Gowalla contact network using Algorithms \ref{['algo:fcf']} and \ref{['algo:spf']} for 100 independent runs in which the disease does not die out in 600 time steps. The parameters are $\beta=0.27, \, \sigma=1/2, \, \gamma=1/7, \,\rho=1/90, \, \lambda_{FP} = 0.1, \, \lambda_{FN} = 0.3$. Light gray: estimates from individual runs. Dark solid: mean estimate averaged over 100 independent runs.
  • Figure 5: Parameter estimation of an SEIRS epidemic model for the Youtube contact network using Algorithms \ref{['algo:fcf']} and \ref{['algo:spf']} for 100 independent runs in which the disease does not die out in 600 time steps. The parameters are $\beta=0.2, \, \sigma=1/3, \, \gamma=1/14, \, \rho=1/180, \, \lambda_{FP} = 0.1, \, \lambda_{FN} = 0.1$. Light gray: estimates from individual runs. Dark solid: mean estimate averaged over 100 independent runs.
  • ...and 9 more figures