Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Non-Negative Universal Differential Equations With Applications in Systems Biology

Maren Philipps, Antonia Körner, Jakob Vanhoefer, Dilan Pathirana, Jan Hasenauer

TL;DR

The paper addresses the issue that universal differential equations (UDEs) can yield negative values for inherently non-negative biological quantities. It introduces non-negative UDEs (nUDEs) with a theoretical guarantee of non-negativity by incorporating a multiplicative factor $\mathbf{N}(\mathbf{x})$ with $\mathbf{N}(\mathbf{0})=0$, and demonstrates this on synthetic Lotka–Volterra dynamics and a real Boehm model. To improve generalisation and interpretability, the authors propose parameter regularisation and output regularisation for the learned component $\mathbf{U}$, showing that these techniques reduce over-fitting and spurious dynamics while preserving predictive accuracy. Through implementation with AMICI/PEtab and multi-start optimisation, the work provides practical guidance on balancing expressivity of the ANN with biological constraints, highlighting that the choice of $\mathbf{N}(\mathbf{x})$ and regularisation strength crucially affects performance. Overall, the study advances biologically meaningful hybrid models by ensuring non-negativity and enabling robust, interpretable learning of unknown mechanisms in systems biology.

Abstract

Universal differential equations (UDEs) leverage the respective advantages of mechanistic models and artificial neural networks and combine them into one dynamic model. However, these hybrid models can suffer from unrealistic solutions, such as negative values for biochemical quantities. We present non-negative UDE (nUDEs), a constrained UDE variant that guarantees non-negative values. Furthermore, we explore regularisation techniques to improve generalisation and interpretability of UDEs.

Non-Negative Universal Differential Equations With Applications in Systems Biology

TL;DR

The paper addresses the issue that universal differential equations (UDEs) can yield negative values for inherently non-negative biological quantities. It introduces non-negative UDEs (nUDEs) with a theoretical guarantee of non-negativity by incorporating a multiplicative factor with , and demonstrates this on synthetic Lotka–Volterra dynamics and a real Boehm model. To improve generalisation and interpretability, the authors propose parameter regularisation and output regularisation for the learned component , showing that these techniques reduce over-fitting and spurious dynamics while preserving predictive accuracy. Through implementation with AMICI/PEtab and multi-start optimisation, the work provides practical guidance on balancing expressivity of the ANN with biological constraints, highlighting that the choice of and regularisation strength crucially affects performance. Overall, the study advances biologically meaningful hybrid models by ensuring non-negativity and enabling robust, interpretable learning of unknown mechanisms in systems biology.

Abstract

Universal differential equations (UDEs) leverage the respective advantages of mechanistic models and artificial neural networks and combine them into one dynamic model. However, these hybrid models can suffer from unrealistic solutions, such as negative values for biochemical quantities. We present non-negative UDE (nUDEs), a constrained UDE variant that guarantees non-negative values. Furthermore, we explore regularisation techniques to improve generalisation and interpretability of UDEs.
Paper Structure (15 sections, 1 theorem, 13 equations, 6 figures)

This paper contains 15 sections, 1 theorem, 13 equations, 6 figures.

Table of Contents

  1. Introduction
  2. Modelling
  3. Mechanistic modelling with ODEs
  4. Machine learning with ANNs
  5. Universal differential equations
  6. Maximum likelihood estimation
  7. Towards biologically meaningful UDEs
  8. Parameter regularisation:
  9. Output regularisation:
  10. Non-negative UDEs
  11. Implementation and Benchmarking
  12. Implementation
  13. Lotka-Volterra model
  14. Boehm model
  15. Discussion

Key Result

Theorem 1

Consider the ODEs IVP, where $\mathbf{f}$ is Lipschitz-continuous. If the non-negative quadrant is invariant under $\mathbf{f}$, i.e. $\left.f_i\right|_{x_i=0} \ge 0\, \forall i \in \{1,\ldots,n_{\mathbf{x}}\}$, then $x_{nUDE,i}\ge 0\, \forall t \ge t_0$ in a nUDE system equation:nude_system with the same $\mathbf{f}$.

Figures (6)

  • Figure 1: Solution characteristics for the Lotka-Volterra model: the number of optimisation runs yielding solutions with negative (orange) or strictly non-negative (gray) values for the predator or prey abundances.
  • Figure 2: Waterfall plot for the Lotka-Volterra model. The objective function value $J$ on the validation data are shown for the 20 best fits on the training data.
  • Figure 3: Fit and prediction for the Lotka-Volterra UDE. The vertical dashed line indicates the training and validation data split. The fits and predictions from the "nUDE; $N(x) = x$" and "nUDE; $N(x) = x$; $\lambda_o=0.01$" models are visually indistinguishable to the UDE, and are not shown.
  • Figure 4: Boehm Scenario 1 comparison between UDE and nUDEs. a) Amount of models that stayed non-negative in their trajectories, and b) distribution of model calibration times per method. Horizontal lines indicate the minimum, medium and maximum.
  • Figure 5: Ensembles of 20 best UDE model variants. a)Scenario 1: ANN with 1 input/2 outputs. Best fits are shown for UDE, nUDE (${\mathbf{N}}({\mathbf{x}})={\mathbf{x}}$) and nUDE (${\mathbf{N}}({\mathbf{x}})=\tanh({\mathbf{x}})$), no regularisation. b)Scenario 2: ANN with 3 inputs/5 outputs. Best fits are shown for the unregularised UDE and a parameter-regularised UDE with $\lambda_p=10$.
  • ...and 1 more figures

Theorems & Definitions (1)

  • Theorem 1