Toward Interpretable and Generalizable AI in Regulatory Genomics

TL;DR

This review analyzes seq2func models that map DNA sequence to regulatory activity and examines how architecture, data, tasks, interpretation, and evaluation shape generalization across perturbations and cellular contexts. It advocates a causal refinement framework where targeted perturbations and continual learning iteratively improve genome-wide models, linking sequence features to mechanistic regulatory rules. The paper outlines strategies for local and global interpretability, causal probing via virtual perturbations, and inherently interpretable representations, while highlighting evaluation challenges posed by distribution shifts. The proposed self-improving genomic AI aims to integrate genome-wide assays, locus-specific perturbations, and biophysical measurements into a feedback loop that enhances generalization and biological discovery, ultimately producing tools usable by experimental biologists.

Abstract

Deciphering how DNA sequence encodes gene regulation remains a central challenge in biology. Advances in machine learning and functional genomics have enabled sequence-to-function (seq2func) models that predict molecular regulatory readouts directly from DNA sequence. These models are now widely used for variant effect prediction, mechanistic interpretation, and regulatory sequence design. Despite strong performance on held-out genomic regions, their ability to generalize across genetic variation and cellular contexts remains inconsistent. Here we examine how architectural choices, training data, and prediction tasks shape the behavior of seq2func models. We synthesize how interpretability methods and evaluation practices have probed learned cis-regulatory organization and highlighted systematic failure modes, clarifying why strong predictive accuracy can fail to translate into robust regulatory understanding. We argue that progress will require reframing seq2func models as continually refined systems, in which targeted perturbation experiments, systematic evaluation, and iterative model updates are tightly coupled through AI-experiment feedback loops. Under this framework, seq2func models become self-improving tools that progressively deepen their mechanistic grounding and more reliably support biological discovery.

Figures (5)

• Figure 1: Standard seq2func workflow. a, (Top) Schematic of gene regulation and functional genomics tracks. (Bottom) A seq2func model takes one-hot–encoded DNA sequence as input, processes it through a feedforward trunk of shared layers, and produces predictions via task-specific heads, which may be single-task or multi-task. b, Schematic of post hoc attribution analysis, in which gradients or related signals are propagated from a selected output track back to the input sequence to generate an attribution map. Collections of attribution maps can be clustered to identify sequence motifs as contribution weight matrices. c, Schematic of an in silico experimentation platform. DNA sequences are perturbed at multiple scales---ranging from single-nucleotide variants to motif-level or cis-regulatory–element perturbations---and evaluated by the model to predict quantitative effect sizes. Perturbations introduced within a locus assess necessity, while elements placed in diverse sequence contexts and averaged assess sufficiency.
• Figure 2: Landscape of seq2func models by genomic receptive field and task breadth. Shown is the number of prediction tasks versus the input receptive field for representative seq2func models. Colors indicate model type: generalist models correspond to multitask architectures (navy), while specialist models correspond to single-task architectures (light blue). Marker size is proportional to the reported parameter count. A gold marker edge indicates models that produce base-pair–aligned predictions. Select models shown include: DeepSea zhou2015predicting, Basset kelley2016basset, Beluga zhou2018deep, Sei chen2022sequence, Basenji2 kelley2018sequential, PromoterAI jaganathan2025predicting, Enformer avsec2021effective, VariantFormer ghosal2025variantformer, Enigma jung2025enigma, Borzoi linder2025predicting, AlphaGenome avsec2025alphagenome, DeepBind alipanahi2015predicting, APARENT2 linder2022deciphering, MPRA-LEGNet agarwal2025massively, DeepSTARR de2022deepstarr, ChromBPNet pampari2025chrombpnet, SpliceAI jaganathan2019predicting, Saluki agarwal2022genetic, Pangolin zeng2022predicting, Puffin-D dudnyk2024sequence, Akita fudenberg2020predicting, and Orca zhou2022sequence.
• Figure 3: Perturbation libraries to probe the cis-regulatory code. Massively parallel reporter assays (MPRAs) introduce dense sequence variation within cis-regulatory elements (CREs) to measure how local nucleotide changes affect regulatory activity. In contrast, CRISPR interference (CRISPRi) libraries target entire CREs, individually or in combination, within their native chromatin context, enabling tests of CRE necessity, CRE–CRE interactions, and long-range or context-dependent regulatory effects.
• Figure 4: Overview of continual learning strategies. a, A conceptual view of parameter space shows how naïve fine-tuning pushes parameters toward the optimal shaded region of each new task, often degrading performance on earlier tasks, whereas continual learning methods encourage movement toward shared optima that preserve prior capabilities. b, Replay-based methods maintain performance over time by mixing past examples with new data during fine-tuning. c, Regularization-based methods, such as Elastic Weight Consolidation, reduce forgetting by constraining updates to parameters deemed important for earlier tasks.
• Figure 5: Overview of causal refinement framework. A seq2func model is first pre-trained on large corpus of genome-wide profiling data to learn broad regulatory patterns across cellular contexts. The model then enters phase two, an iterative loop in which targeted perturbation assays, such as MPRA and CRISPRi, are designed using active learning strategies. These experiments generate new measurements that are integrated back into the model through continual learning. Each iteration incrementally refines the model's understanding of cis-regulatory mechanisms one locus at a time, progressively improving broader regulatory knowledge across the genome.