Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Automatic and Structure-Aware Sparsification of Hybrid Neural ODEs

Bob Junyi Zou, Lu Tian

TL;DR

Hybrid neural ODEs address data-scarce healthcare challenges by integrating mechanistic dynamics with neural components, but risk overfitting due to large latent state spaces. The authors propose Hybrid Graph Sparsification (HGS) for MNODEs, a three-step pipeline that combines domain-informed graph refinement with gradient-based regularization to prune edges and states while preserving mechanistic structure. Specifically, they (i) merge strongly connected components into a relaxed DAG, (ii) augment essential pathways with partial transitive closures, and (iii) apply a combined $L_1$/$L_2$ penalty on edge weights $W$ and decoder parameters $\Theta$, all within an encoder–decoder MNODE framework trained by forward-Euler discretization. Experiments on synthetic data and real-world T1D glucose dynamics (UVA-Padova) show improved predictive accuracy and robustness with sparser, more interpretable models, demonstrating effective hybrid model reduction in healthcare with limited data.

Abstract

Hybrid neural ordinary differential equations (neural ODEs) integrate mechanistic models with neural ODEs, offering strong inductive bias and flexibility, and are particularly advantageous in data-scarce healthcare settings. However, excessive latent states and interactions from mechanistic models can lead to training inefficiency and over-fitting, limiting practical effectiveness of hybrid neural ODEs. In response, we propose a new hybrid pipeline for automatic state selection and structure optimization in mechanistic neural ODEs, combining domain-informed graph modifications with data-driven regularization to sparsify the model for improving predictive performance and stability while retaining mechanistic plausibility. Experiments on synthetic and real-world data show improved predictive performance and robustness with desired sparsity, establishing an effective solution for hybrid model reduction in healthcare applications.

Automatic and Structure-Aware Sparsification of Hybrid Neural ODEs

TL;DR

Hybrid neural ODEs address data-scarce healthcare challenges by integrating mechanistic dynamics with neural components, but risk overfitting due to large latent state spaces. The authors propose Hybrid Graph Sparsification (HGS) for MNODEs, a three-step pipeline that combines domain-informed graph refinement with gradient-based regularization to prune edges and states while preserving mechanistic structure. Specifically, they (i) merge strongly connected components into a relaxed DAG, (ii) augment essential pathways with partial transitive closures, and (iii) apply a combined / penalty on edge weights and decoder parameters , all within an encoder–decoder MNODE framework trained by forward-Euler discretization. Experiments on synthetic data and real-world T1D glucose dynamics (UVA-Padova) show improved predictive accuracy and robustness with sparser, more interpretable models, demonstrating effective hybrid model reduction in healthcare with limited data.

Abstract

Hybrid neural ordinary differential equations (neural ODEs) integrate mechanistic models with neural ODEs, offering strong inductive bias and flexibility, and are particularly advantageous in data-scarce healthcare settings. However, excessive latent states and interactions from mechanistic models can lead to training inefficiency and over-fitting, limiting practical effectiveness of hybrid neural ODEs. In response, we propose a new hybrid pipeline for automatic state selection and structure optimization in mechanistic neural ODEs, combining domain-informed graph modifications with data-driven regularization to sparsify the model for improving predictive performance and stability while retaining mechanistic plausibility. Experiments on synthetic and real-world data show improved predictive performance and robustness with desired sparsity, establishing an effective solution for hybrid model reduction in healthcare applications.

Paper Structure

This paper contains 74 sections, 75 equations, 13 figures, 3 tables, 10 algorithms.

Table of Contents

  1. Introduction
  2. Methodology
  3. Related work
  4. Experiments
  5. Broader impact and limitations
  6. Additional Details about HGS
  7. Illustration of HGS
  8. Proof for Group LASSO Equivalence
  9. APPENDIX: Real-World Data Pre-processing
  10. Selection
  11. Features, Units and Interpolation
  12. Interpolating Basal Flow Rate
  13. Interpolating Bolus Insulin Rate
  14. Interpolating Carbohydrate Intake Rate
  15. Interpolating Heart Rate and Step Count
  16. ...and 59 more sections

Figures (13)

  • Figure 1: Comparison across different evaluation settings. GD is omitted for comprehensive initial graph as training takes an unreasonable amount of time. Models are sorted from best to worst.
  • Figure 2: Combined Results for Real-world Experiments. Models are sorted from best to worst.
  • Figure 3: Step 1 and 2 of the Hybrid Graph Sparsification Algorithm
  • Figure 4: An illustration of raw and pre-process data of one of the exercise instances.
  • Figure 5: An illustration of the mechanistic vs true graphs used in the synthetic experiments
  • ...and 8 more figures