ARMAX identification of low rank graphical models

TL;DR

This paper tackles the problem of identifying low-rank stochastic processes from noisy measurements by exploiting a sparse-plus-low-rank (S+L) structure in latent-variable graphical models. It proposes a two-stage framework: (i) use maximum-entropy covariance extension to estimate a time-series model for the low-rank latent variable $y_l$, and (ii) perform a maximum-likelihood ARMAX graphical estimation to recover the deterministic relation $H(z)$ linking $y_l$ to the observed data, with identifiability and consistency proven under suitable conditions. The approach yields a consistent ML estimator and an accurate denoising filter, demonstrated via simulations where it significantly outperforms traditional low-rank identification that ignores measurement noise. The results have practical implications for extracting meaningful, reduced-dimension structure from high-dimensional noisy systems and for identifying key latent factors directly from data.

Abstract

In large-scale systems, complex internal relationships are often present. Such interconnected systems can be effectively described by low rank stochastic processes. When identifying a predictive model of low rank processes from sampling data, the rank-deficient property of spectral densities is often obscured by the inevitable measurement noise in practice. However, existing low rank identification approaches often did not take noise into explicit consideration, leading to non-negligible inaccuracies even under weak noise. In this paper, we address the identification issue of low rank processes under measurement noise. We find that the noisy measurement model admits a sparse plus low rank structure in latent-variable graphical models. Specifically, we first decompose the problem into a maximum entropy covariance extension problem, and a low rank graphical estimation problem based on an autoregressive moving-average with exogenous input (ARMAX) model. To identify the ARMAX low rank graphical models, we propose an estimation approach based on maximum likelihood. The identifiability and consistency of this approach are proven under certain conditions. Simulation results confirm the reliable performance of the entire algorithm in both the parameter estimation and noisy data filtering.

Figures (8)

• Figure 1: The structure block diagram of a special feedback model \ref{['eq:specialFB']}.
• Figure 2: Structure block diagrams of measurement model \ref{['eq:meas_model']} and low rank graphical model \ref{['eq:zeta=yl+e']}. By utilizing the low rank characteristic, measurement process $\zeta(t)$ only depends on an $l$-dimensional low rank latent variable $y_l(t)$, rather than the higher-dimensional process $y(t)$. $y_l(t)$ can be seen as the key factor of the hidden low rank process $y(t)$.
• Figure 3: The topology of a low rank graphical model \ref{['eq:zeta=yl+e']}, with $\zeta(t)$, $y(t)$, $y_l(t)$ of dimension $5$, $5$, $2$ respectively.
• Figure 4: Example 1: the estimation in one MC experiment of $y_l(t)=[y_{(5)}(t),~y_{(6)}(t),~y_{(7)}(t)]'$ in time period $t=310,\cdots, 350$, and of $y_m(t)=[y_{(1)},~y_{(2)},~y_{(3)},~y_{(4)}]'$ in time period $t=320,\cdots, 350$. The blue solid line denotes the real data of $y_{(i)}(t)$. The green dotted line and the pink dotted-dashed line denotes the estimated $y_{(i)}(t)$ by traditional low rank identification and the low rank graphical identification, respectively.
• Figure 5: Example 1: coherence of $y_l(t)$ calculated by real or estimated AR (graphical) models in MC experiments.

Theorems & Definitions (13)

• Theorem 1
• proof
• Theorem 2: Identifiability of low rank ARMAX graphical model
• proof
• Proposition 3: Jacobin and Hessian
• proof
• Theorem 4
• Theorem 5: Consistency of the maximum likelihood estimate
• proof
• Theorem 6: Identifiability of ARMAX model, Chen93