Efficient Approximation of Volterra Series for High-Dimensional Systems
Navin Khoshnan, Claudia K Petritsch, Bryce-Allen Bagley
TL;DR
The paper tackles the curse of dimensionality in identifying high-dimensional nonlinear dynamical systems via Volterra series. It introduces Tensor Head Averaging (THA), an ensemble of localized MVMALS heads trained on small input subsets and averaged to predict outputs, thereby replacing full-dimension learning with subset-based modeling. The authors derive observable, finite-sample error bounds and an exact geometric decomposition that reveals a baking gain: optimization over localized heads implicitly compensates for omitted dynamics, yielding accuracy beyond simple truncation. THA achieves a dramatic reduction in computational complexity, from a dependence on the full input dimension to a dependence on the much smaller subset size, enabling scalable identification of previously intractable high-dimensional systems while preserving interpretability of Volterra kernels. The work provides a principled framework for scalable Volterra identification with theoretical guarantees and motivates future empirical validation and adaptive subset strategies.
Abstract
The identification of high-dimensional nonlinear dynamical systems via the Volterra series has significant potential, but has been severely hindered by the curse of dimensionality. Tensor Network (TN) methods such as the Modified Alternating Linear Scheme (MVMALS) have been a breakthrough for the field, offering a tractable approach by exploiting the low-rank structure in Volterra kernels. However, these techniques still encounter prohibitive computational and memory bottlenecks due to high-order polynomial scaling with respect to input dimension. To overcome this barrier, we introduce the Tensor Head Averaging (THA) algorithm, which significantly reduces complexity by constructing an ensemble of localized MVMALS models trained on small subsets of the input space. In this paper, we present a theoretical foundation for the THA algorithm. We establish observable, finite-sample bounds on the error between the THA ensemble and a full MVMALS model, and we derive an exact decomposition of the squared error. This decomposition is used to analyze the manner in which subset models implicitly compensate for omitted dynamics. We quantify this effect, and prove that correlation between the included and omitted dynamics creates an optimization incentive which drives THA's performance toward accuracy superior to a simple truncation of a full MVMALS model. THA thus offers a scalable and theoretically grounded approach for identifying previously intractable high-dimensional systems.