Fourier-enhanced Neural Networks For Systems Biology Applications

TL;DR

This work introduces the System-Biology Fourier-enhanced Neural Network (SB-FNN), a Fourier-based neural architecture designed to efficiently solve initial-value problems in complex systems biology. By embedding Fourier layers, and employing a per-layer adaptive activation function alongside a variance-based oscillation penalty, SB-FNN better captures oscillatory and multiscale dynamics than conventional PINNs. Across six models, including repressilator variants, SIR with age structure, and 1D/2D Turing patterns, SB-FNN demonstrates higher accuracy (lower N-MSE) and faster training, with ablation studies confirming the value of both the adaptive activations and the variance constraint. These results suggest SB-FNN as a practical alternative to PINN for large-scale, oscillatory, and diffusion-influenced biological systems, with potential for further extensions to stiff dynamics and inverse modeling.

Abstract

In the field of systems biology, differential equations are commonly used to model biological systems, but solving them for large-scale and complex systems can be computationally expensive. Recently, the integration of machine learning and mathematical modeling has offered new opportunities for scientific discoveries in biology and health. The emerging physics-informed neural network (PINN) has been proposed as a solution to this problem. However, PINN can be computationally expensive and unreliable for complex biological systems. To address these issues, we propose the Fourier-enhanced Neural Networks for systems biology (SB-FNN). SB-FNN uses an embedded Fourier neural network with an adaptive activation function and a cyclic penalty function to optimize the prediction of biological dynamics, particularly for biological systems that exhibit oscillatory patterns. Experimental results demonstrate that SB-FNN achieves better performance and is more efficient than PINN for handling complex biological models. Experimental results on cellular and population models demonstrate that SB-FNN outperforms PINN in both accuracy and efficiency, making it a promising alternative approach for handling complex biological models. The proposed method achieved better performance on six biological models and is expected to replace PINN as the most advanced method in systems biology.

Figures (13)

• Figure 1: Fourier-enhanced Neural Network for dynamic systems biology model prediction. (a) The SB-FNN includes an input fully-connected neural network $\mathcal{U}$, $l$ Fourier layers, as well as an output fully connected neural network. (b) The loss function in the SB-FNN model is composed of four different loss components $Loss_{o}$ (initial condition), $Loss_{f}$ (residual loss), $Loss_{b}$ (boundary condition) and $Loss_{p}$ (physical penalty), each with its corresponding hyperparameter $\lambda_o$, $\lambda_f$, $\lambda_b$ and $\lambda_p$.
• Figure 2: Adaptive activation function. The adaptive weights, denoted as $w$, are generated by applying a Softmax operator to a trainable tensor vector, represented as $r$. In SB-FNN, Tanh, ReLU, Softplus, ELU, GELU, and Sin are chosen as the activation function candidates, as they have shown excellent performance across various models and tasks.
• Figure 3: Plot of the variance penalty function $\Phi\left(x\right)$. The parameter $\alpha$ is utilized to manage the position of the symmetry center of the function along the x-axis, and the parameter $\tau$ is used to control the rate of change of the function around $x=\alpha$. A larger value of $\tau$ results in a faster decrease of the function from $y=1$ to $y=0$ in the neighborhood of $x=\alpha$.
• Figure 4: Prediction of the Repressilator: protein only model. $P_{i}$ ($i=$$lacI$, $tetR$, or $cI$) represents the concentrations of the repressor-proteins of $lacI$, $tetR$, and $cI$, respectively. The model parameters are $\beta = 10$ and $n=3$.
• Figure 5: Prediction of the Repressilator: mRNA and protein model. $P_{i}$ and $M_{i}$ ($i=$$lacI$, $tetR$, or $cI$) represents the repressor-proteins concentrations and mRNA concentrations of $lacI$, $tetR$, and $cI$, respectively. The model parameters are $\beta = 10$, $n=3$, $\alpha=10$ and $\alpha_0=1e^{-5}$.