Are Statistical Methods Obsolete in the Era of Deep Learning?

TL;DR

The paper investigates whether statistically principled methods remain relevant in the era of deep learning by comparing PINN, a physics-informed neural network, with MAGI, a Bayesian Gaussian-process-based ODE inference method, on SEIR and Lorenz models under sparse and noisy data. MAGI generally offers robust parameter inference, accurate trajectory reconstruction, and credible uncertainty quantification, particularly when data are limited or components are unobserved, while PINN’s performance is highly sensitive to hyperparameters and can struggle with out-of-sample forecasts in chaotic or partially observed settings. The results advocate for the continued value of statistical approaches in scientific modeling and highlight the potential of integrating probabilistic ODE inference with neural methods to leverage the strengths of both paradigms. Overall, the work emphasizes forecasting reliability, mechanistic fidelity, and uncertainty quantification as key benefits of MAGI, even as deep learning advances continue to transform data-driven modeling.

Abstract

In the era of AI, neural networks have become increasingly popular for modeling, inference, and prediction, largely due to their potential for universal approximation. With the proliferation of such deep learning models, a question arises: are leaner statistical methods still relevant? To shed insight on this question, we employ the mechanistic nonlinear ordinary differential equation (ODE) inverse problem as a testbed, using physics-informed neural network (PINN) as a representative of the deep learning paradigm and manifold-constrained Gaussian process inference (MAGI) as a representative of statistically principled methods. Through case studies involving the SEIR model from epidemiology and the Lorenz model from chaotic dynamics, we demonstrate that statistical methods are far from obsolete, especially when working with sparse and noisy observations. On tasks such as parameter inference and trajectory reconstruction, statistically principled methods consistently achieve lower bias and variance, while using far fewer parameters and requiring less hyperparameter tuning. Statistical methods can also decisively outperform deep learning models on out-of-sample future prediction, where the absence of relevant data often leads overparameterized models astray. Additionally, we find that statistically principled approaches are more robust to accumulation of numerical imprecision and can represent the underlying system more faithful to the true governing ODEs.

Figures (21)

• Figure 1: Trajectory reconstruction and prediction by PINN and MAGI for the SEIR model in the fully observed case. The dots show one sample dataset (out of 100). The dashed black lines give the true curves, which are to be identified. Each solid blue curve is the estimate from one dataset. The red dashed vertical line separates the in-sample observation period from the future forecasting period. Top row: MAGI estimates. Lower two rows: PINN estimates, with $\lambda = 10$ and $\lambda = 1000$, respectively. Each solid blue curve is the estimate from one dataset.
• Figure 2: Boxplots showing the RMSE on the logarithm of the SEIR system components across 100 datasets in the fully observed case. Lower value indicates better performance. The y-axis is displayed in the logarithmic scale for better visualization. Top row: in-sample trajectory reconstruction (Equation \ref{['eq:in-sample RMSE']}); bottom row: future forecasting (Equation \ref{['eq:pred RMSE']}). The three columns correspond to three system components: $E$, $I$, and $R$. In each panel, the leftmost boxplot is for MAGI, and the remaining boxplots are for PINN under different hyperparameters $\lambda$; the dashed vertical line separates MAGI and PINN.
• Figure 3: Boxplots showing the absolute errors in parameter estimation for PINN and MAGI across 100 datasets in the fully observed case. Lower value indicates better performance. The y-axis is displayed in the logarithmic scale for better visualization. Top row: The errors for the original parameters $\beta$, $\gamma$, and $\sigma$. Bottom row: The errors for $R_0$, peak timing, and peak intensity -- our three quantities of interest. In each panel, the leftmost blue boxplot is for MAGI, and the remaining red boxplots are for PINN under different $\lambda$ settings; the dashed vertical line separates MAGI and PINN results.
• Figure 4: Trajectory reconstruction and prediction by PINN and MAGI for the SEIR model in the missing component case, where the $E$ component is unobserved. The dots show one sample dataset (out of 100). The legend and layout of this figure are identical to Figure \ref{['fig:seir_pinn_full_subset']}; see the caption there.
• Figure 5: Boxplots showing the RMSE on the logarithm of the SEIR system components across 100 datasets in the missing $E$ component case. The legend and layout of this figure are identical to Figure \ref{['fig:boxplot_seir_traj_err_full_logscale']}; see the caption there.