Analytical Gradient and Hessian Evaluation for System Identification using State-Parameter Transition Tensors

TL;DR

The paper tackles the problem of identifying unknown system parameters and initial conditions in nonlinear dynamical systems by deriving analytical gradient and Hessian information. It achieves this by formulating a generalized ODE-based framework that computes state and parameter transition tensors (including higher-order variants) in Einstein notation, and then expresses the cost’s derivatives in terms of these tensors. The authors demonstrate that the analytical gradient and Hessian enable robust, second-order optimization, outperforming finite-difference approaches and a commercial MATLAB toolbox on two benchmark problems (the Silver box and the Two-tank), with GOF values around $96\%$ and $79\%$, respectively. This approach offers a principled, numerically stable pathway for parameter and initial-condition identification in complex dynamical systems, with potential broad impact on system identification workflows requiring reliable second-order information.

Abstract

In this work, the Einstein notation is utilized to synthesize state and parameter transition matrices, by solving a set of ordinary differential equations. Additionally, for the system identification problem, it has been demonstrated that the gradient and Hessian of a cost function can be analytically constructed using the same matrix and tensor metrics. A general gradientbased optimization problem is then posed to identify unknown system parameters and unknown initial conditions. Here, the analytical gradient and Hessian of the cost function are derived using these state and parameter transition matrices. The more robust performance of the proposed method for identifying unknown system parameters and unknown initial conditions over an existing conventional quasi-Newton method-based system identification toolbox (available in MATLAB) is demonstrated by using two widely used benchmark datasets from real dynamic systems. In the existing toolbox, gradient and Hessian information, which are derived using a finite difference method, are more susceptible to numerical errors compared to the analytical approach presented. Keywords: Gradient-based Optimization, Transition matrix and tensors, Gradient and Hessian, System identification.

Figures (3)

• Figure 1: Comparison of model $y$ data and observation $y$ data over the training data set.
• Figure 2: Comparison of model $y$ data and observation $y$ data over the validation data set.
• Figure 3: Comparison of model $y$ data and observation $y$ data over the validation data set.