An updated look on the convergence and consistency of data-driven dynamical models
Kristian Løvland, Bjarne Grimstad, Lars Struen Imsland
TL;DR
The paper addresses convergence and consistency for nonlinear probabilistic sequence models used to learn controlled dynamical systems under maximum likelihood estimation. It proves that, under stability-like forgetting and regularity conditions, the log-likelihood objective $L_T(\theta)$ converges uniformly to a limit $\overline{L}(\theta)$ and the ML estimator satisfies $\hat{\theta}_T \to \arg\max_{\theta} \overline{L}(\theta)$. A consistency result for $s$-th order Markov properties shows ML converges to a KL-divergence minimizer, and with full support yields recovery of the true conditional $p(Y|\Phi)$. Two illustrative examples—linear system identification and finite-state Markov chains—demonstrate the theory’s applicability to both continuous and discrete observation spaces. Overall, the work extends classical PE identification to probabilistic sequence models with long memory and general observation spaces, informing design and analysis of data-driven dynamical models in control and RL contexts.
Abstract
Deep sequence models are receiving significant interest in current machine learning research. By representing probability distributions that are fit to data using maximum likelihood estimation, such models can model data on general observation spaces (both continuous and discrete-valued). Furthermore, they can be applied to a wide range of modelling problems, including modelling of dynamical systems which are subject to control. The problem of learning data-driven models of systems subject to control is well studied in the field of system identification. In particular, there exist theoretical convergence and consistency results which can be used to analyze model behaviour and guide model development. However, these results typically concern models which provide point predictions of continuous-valued variables. Motivated by this, we derive convergence and consistency results for a class of nonlinear probabilistic models defined on a general observation space. The results rely on stability and regularity assumptions, and can be used to derive consistency conditions and bias expressions for nonlinear probabilistic models of systems under control. We illustrate the results on examples from linear system identification and Markov chains on finite state spaces.