Manifold meta-learning for reduced-complexity neural system identification

TL;DR

The paper tackles data and computation bottlenecks in neural system identification by meta-learning a low-dimensional parameter manifold within an over-parameterized base network. It replaces bilevel inner-loop optimization with an encoder-decoder framework that maps datasets to reduced coordinates, enabling fast, data-efficient fitting across a family of related dynamical systems. Applied to the Bouc-Wen benchmark, the approach demonstrates strong performance in low-data regimes and shows that a 20-dimensional manifold can achieve competitive accuracy with substantially reduced training effort, while providing a practical initialization for reduced-complexity SYSID. The work suggests promising extensions to incorporate physical priors and probabilistic uncertainty, offering a scalable path for robust, physics-informed reduced models in complex dynamical identification tasks.

Abstract

System identification has greatly benefited from deep learning techniques, particularly for modeling complex, nonlinear dynamical systems with partially unknown physics where traditional approaches may not be feasible. However, deep learning models often require large datasets and significant computational resources at training and inference due to their high-dimensional parameterizations. To address this challenge, we propose a meta-learning framework that discovers a low-dimensional manifold within the parameter space of an over-parameterized neural network architecture. This manifold is learned from a meta-dataset of input-output sequences generated by a class of related dynamical systems, enabling efficient model training while preserving the network's expressive power for the considered system class. Unlike bilevel meta-learning approaches, our method employs an auxiliary neural network to map datasets directly onto the learned manifold, eliminating the need for costly second-order gradient computations during meta-training and reducing the number of first-order updates required in inference, which could be expensive for large models. We validate our approach on a family of Bouc-Wen oscillators, which is a well-studied nonlinear system identification benchmark. We demonstrate that we are able to learn accurate models even in small-data scenarios.

Figures (4)

• Figure 1: The autoencoder architecture for reduced-complexity architecture learning.
• Figure 2: Eigenvalues of the Hessian of the full-order training loss.
• Figure 3: Full and reduced-order models: test FIT vs. training sequence length $L$.
• Figure 4: Training time vs. training sequence length $L$.

Theorems & Definitions (2)

• Remark 1
• Remark 2