Universal Differential Equations as a Common Modeling Language for Neuroscience

TL;DR

The paper advocates universal differential equations (UDEs) as a unifying language for neuroscience, bridging mechanistic, phenomenological, and data-driven perspectives by treating differential equations as parameterizable, differentiable objects that can be trained with scalable AI techniques. It presents a parameterized stochastic differential equation formulation $dx(t) = \mu_{\alpha}(x(t),u(t))dt + \sigma_{\beta}(x(t),u(t))dW(t)$ with learnable $\theta=(\alpha,\beta)$, and outlines a four-module neural system identification pipeline (stimulus encoder, recognition model, process model, observation model) trained via variational inference. The authors outline a spectrum of UDE configurations from white-box to fully neural (drift and diffusion) models, discuss practical training considerations, and highlight opportunities in explaining cognition, neural control, decoding, and normative modelling, while also acknowledging challenges in gradient-through-solvers and noise modelling. By providing concrete modeling recipes and a principled probabilistic framework, the work aims to establish UDEs as a robust, scalable, and interpretable foundation for multi-scale neural dynamics across diverse neuroengineering applications.

Abstract

The unprecedented availability of large-scale datasets in neuroscience has spurred the exploration of artificial deep neural networks (DNNs) both as empirical tools and as models of natural neural systems. Their appeal lies in their ability to approximate arbitrary functions directly from observations, circumventing the need for cumbersome mechanistic modeling. However, without appropriate constraints, DNNs risk producing implausible models, diminishing their scientific value. Moreover, the interpretability of DNNs poses a significant challenge, particularly with the adoption of more complex expressive architectures. In this perspective, we argue for universal differential equations (UDEs) as a unifying approach for model development and validation in neuroscience. UDEs view differential equations as parameterizable, differentiable mathematical objects that can be augmented and trained with scalable deep learning techniques. This synergy facilitates the integration of decades of extensive literature in calculus, numerical analysis, and neural modeling with emerging advancements in AI into a potent framework. We provide a primer on this burgeoning topic in scientific machine learning and demonstrate how UDEs fill in a critical gap between mechanistic, phenomenological, and data-driven models in neuroscience. We outline a flexible recipe for modeling neural systems with UDEs and discuss how they can offer principled solutions to inherent challenges across diverse neuroscience applications such as understanding neural computation, controlling neural systems, neural decoding, and normative modeling.

Figures (3)

• Figure 1: Universal differential equations a) A schematic illustration of a universal differential equation. The vector field of the differential equation is defined via either an existing model from the literature, or a differentiable universal approximator (e.g. a neural network) or a combination of both. The numerical solver is an SDE-compatible numerical solver, which takes in the initial condition $x_0$, a geometric Brownian motion generator $\Delta W$, the forcing signal $u$, the functions defining the vector fields of the SDE $\mu$ and $\sigma$, along with their parameters $\theta$. The numerical solver then computes the solution at time $t$. The parameters of the differential equation can then be trained either via automatic differentiation or using adjoints methods. This setup enables the use of a UDE either as universal function approximator on their own or as a part (layer) in a differentiable computational graph. b) The formulation of a UDE encompasses a spectrum of modeling techniques from white-box traditional models to fully data-driven black box models. This flexibility can foster interoperability between different methodological efforts, provide solid theoretical background to face multifaceted challenges in neuroscience modelling across scales and applications, and offer a principled approach to balance between data adaptability and scientific rationale in model development.
• Figure 2: Framework for neural system identification a) Shows the forward pass (generative mode) during the encoding of a (high-dimensional) stimulus $v$ into neurobehavioral observations $y$. This is done through a fully differentiable graph, which consists of i) a stimulus encoder to encode the stimulus into a lower dimensional continuous representation, ii) a recognition model to infer the hidden initial state $x_0$, iii) a latent dynamics model to model the temporal evolution of the dynamics, and iv) an observation model to map the latent states into observations. b) Illustrates the formulation of the informed stimulus encoder which is tasked with learning a lower dimensional continuous representation $u$ from the discrete (high-dimensional) stimulus signal $v$. c) Illustrates examples of modality-specific observation models to map the latent process into neurobehavioral measurements.
• Figure 3: UDE-based models across different applications in neurosciencea: UDEs can be used to represent the underlying latent dynamical system to understand how neural dynamics give rise to computations and ultimately behaviour. b: UDEs can be used for model-based closed loop control of neural systems. c: UDE models trained for neural encoding can also be leveraged for neural decoding of the stimulus from neural observations. d: UDE models can be employed for capturing population-average neural dynamics utilized for patients stratification in a normative modelling framework.