No Foundations without Foundations -- Why semi-mechanistic models are essential for regulatory biology

TL;DR

The paper argues that robust foundation models for regulatory biology require semi-mechanistic, first-principles frameworks that couple perturbation design with dynamical, mechanistic reasoning. By formalizing perturb-seq experiments with a non-commutative operator F that captures perturbations, media, and time, the authors connect experimental design to learnable models while explicitly addressing differentiating vs non-differentiating cells and measurement artefacts. They show how single-cell measurements map latent cell states to observed data through a measurement operator with batch and dropout considerations, and they situate their approach within VAEs and causal modelling to illustrate connections to established ML methods. A proof-of-principle using an NODE model and a steady-state loss demonstrates improved predictive performance, supporting the claim that incorporating dynamical and biological structure enhances generalization. The work advocates for standardized experiments and first-principles modeling to enable scalable, interpretable, and clinically relevant regulatory biology models.

Abstract

Despite substantial efforts, deep learning has not yet delivered a transformative impact on elucidating regulatory biology, particularly in the realm of predicting gene expression profiles. Here, we argue that genuine "foundation models" of regulatory biology will remain out of reach unless guided by frameworks that integrate mechanistic insight with principled experimental design. We present one such ground-up, semi-mechanistic framework that unifies perturbation-based experimental designs across both in vitro and in vivo CRISPR screens, accounting for differentiating and non-differentiating cellular systems. By revealing previously unrecognised assumptions in published machine learning methods, our approach clarifies links with popular techniques such as variational autoencoders and structural causal models. In practice, this framework suggests a modified loss function that we demonstrate can improve predictive performance, and further suggests an error analysis that informs batching strategies. Ultimately, since cellular regulation emerges from innumerable interactions amongst largely uncharted molecular components, we contend that systems-level understanding cannot be achieved through structural biology alone. Instead, we argue that real progress will require a first-principles perspective on how experiments capture biological phenomena, how data are generated, and how these processes can be reflected in more faithful modelling architectures.

Figures (5)

• Figure 1: Illustration of abstracted phases within a perturb-seq experiment: application of a genetic perturbation, $P$; a change in a media condition, $M$; and the culturing of cells over time, $W$. In panel (A.) we provide a typical wet lab illustration, and in (B.) a branching process illustration with $(n_{\text{\sc p}}+1)(n_{\text{\sc m}}+1)$ total unique branches.
• Figure 2: Illustration of haematopoetic stem cells differentiating into myeloid, erythroid, and lymphoid lineages. In panel (A.) we mark each cell with its corresponding pseudotime value, and in (B.) we label each point by estimated cell type. Inset, we see the distribution of different knockout populations along the trajectory.
• Figure 3: (A.) High level causal graph where each contextual variable potentially acts on all genes within $X$ or some subset. (B.) Gene-level causal graph where $\mathbf{x} = (x_1, \dots, x_{n_{\text{\sc g}}})$ are gene counts and perturbations (e.g., $\text{p}_{\gamma}$) are parameters.
• Figure 4: Mean squared error curve on the test set for numerical proof of principle for the modified train loss function in Section \ref{['sec:attractors']}. Experiment is repeated $100$ times with different random seeds for each loss function.
• Figure 5: Absolute LFC between sequential days in the ishikawa2023renge dataset.