Convergence in On-line Learning of Static and Dynamic Systems
Torbjörn Wigren, Ruoqi Zhang, Per Mattsson
TL;DR
The paper analyzes the convergence properties of a recursive ADAM algorithm applied to nonlinear system identification with a structured RNN model. Using averaging theory, it derives the associated ODEs and proves that, under near-standard hyper-parameters, the asymptotic updating direction matches that of a diagonally power-normalized stochastic gradient, while with internal filtering turned off it matches a sign-sign SGD. It further shows global convergence to an invariant set containing parameter vectors that realize the true input-output behavior, under standard regularity conditions and symmetry assumptions for the sign-sign regime. A nonlinear dynamic model is proposed to embed structure in RNNs, and a Monte-Carlo cruise-control study validates the theoretical results and demonstrates the practical viability of the structured-RNN approach. The findings indicate potential performance advantages of ADAM over conventional normalized gradient methods in online learning of static and dynamic systems.
Abstract
The paper derives analytical expressions for the asymptotic average updating direction of the adaptive moment generation (ADAM) algorithm when applied to recursive identification of nonlinear systems. It is proved that the standard hyper-parameter setting results in the same asymptotic average updating direction as a diagonally power normalized stochastic gradient algorithm. With the internal filtering turned off, the asymptotic average updating direction is instead equivalent to that of a sign-sign stochastic gradient algorithm. Global convergence to an invariant set follows, where a subset of parameters contain those that give a correct input-output description of the system. The paper also exploits a nonlinear dynamic model to embed structure in recurrent neural networks. A Monte-Carlo simulation study validates the results.