Comparison of neural network training strategies for the simulation of dynamical systems

TL;DR

Problem: training strategy choice (parallel vs series-parallel) impacts long-horizon predictive accuracy in neural network models of dynamical systems. Approach: systematic empirical comparison across five architectures on two real-world datasets (pneumatic valve and industrial robot), with evaluation via NRMSE and free-running simulation. Findings: parallel training consistently outperforms series-parallel across architectures and tasks, suggesting it should be the default for simulation and forecasting; the study also clarifies terminology and links to system identification. Implications: improves reliability of data-driven simulators in control and robotics and motivates further exploration of curriculum learning, online adaptation, and physics-informed losses.

Abstract

Neural networks have become a widely adopted tool for modeling nonlinear dynamical systems from data. However, the choice of training strategy remains a key design decision, particularly for simulation tasks. This paper compares two predominant strategies: parallel and series-parallel training. The conducted empirical analysis spans five neural network architectures and two examples: a pneumatic valve test bench and an industrial robot benchmark. The study reveals that, even though series-parallel training dominates current practice, parallel training consistently yields better long-term prediction accuracy. Additionally, this work clarifies the often inconsistent terminology in the literature and relate both strategies to concepts from system identification. The findings suggest that parallel training should be considered the default training strategy for neural network-based simulation of dynamical systems.

Figures (3)

• Figure 1: Information flows for parallel training \ref{['eq:parallel']} and series-parallel training \ref{['eq:series-parallel']}.
• Figure 2: Predicted pressure $p$ and plunger position $s$ for the pneumatic valve system on one representative test trajectory. The top two plots show the applied input voltages $u_1$ and $u_2$. The bottom plots compare the outputs of the best-performing parallel-trained model (p-RNN) and series-parallel-trained model (sp-RNN) against measured data (black dashed lines).
• Figure 3: Predicted joint trajectories $q_1, \dots, q_6$ for the industrial robot over the full test sequence. Measured positions (dashed black lines) are compared against the outputs of the best-performing parallel-trained model (p-RNN) and series-parallel-trained model (sp-TCN), as identified in Tab. \ref{['tab:nrmse_all']}. Note that the desired trajectory repeats around 180s due to the structure of the test set.