Hybrid Modeling Design Patterns

TL;DR

Hybrid modeling seeks to combine first-principles physics with data-driven learning to overcome limitations of each approach. The paper introduces four base design patterns and two composition patterns, formalizes a block-diagram notation with data-driven blocks $D$ and physics-based blocks $P$, and demonstrates how to assemble complex hybrids using $H(x)=D(x)+P(x)$ or $H(x)=P(x, D(x))$. Key contributions include the delta model, physics-based preprocessing, feature learning, and physical constraints, plus recurring and hierarchical composition patterns, all illustrated across climate, engineering, and physics domains. The framework improves data efficiency, extrapolation, and interpretability by embedding physical structure and prior knowledge into data-driven components and provides a modular blueprint for cross-domain hybrid modeling. Overall, these design patterns enable reusable, scalable approaches for integrating domain knowledge with data-driven insights across diverse applications.

Abstract

Design patterns provide a systematic way to convey solutions to recurring modeling challenges. This paper introduces design patterns for hybrid modeling, an approach that combines modeling based on first principles with data-driven modeling techniques. While both approaches have complementary advantages there are often multiple ways to combine them into a hybrid model, and the appropriate solution will depend on the problem at hand. In this paper, we provide four base patterns that can serve as blueprints for combining data-driven components with domain knowledge into a hybrid approach. In addition, we also present two composition patterns that govern the combination of the base patterns into more complex hybrid models. Each design pattern is illustrated by typical use cases from application areas such as climate modeling, engineering, and physics.

Figures (6)

• Figure 1: A block diagram for a hybrid modeling design pattern consists of computational blocks (\ref{['fig:block']}), indicating model components that involve computation, and arrows (\ref{['fig:arrow']}) indicating the flow of data and intermediate computational results. For example, the arrow in \ref{['fig:simple_bd']} indicates that the result of the block $\mathcal{B}_1$ is fed as an input into the computational block $\mathcal{B}_2$.
• Figure 2: Most of the hybrid modeling design patterns can be communicated through block diagrams.
• Figure 3: Evaluation of different methods on a toy accelerometer set-up. From top to bottom: Predictions from a (a) Van der Pol oscillator ($P(t)$), (b) Gaussian Process ($D(t)$) and (c) hybrid model combining both approaches according to the delta model ($H(t) = P(t) + D(t)$). Training data is shown in blue, test data in red. We can observe that the Van der Pol oscillator cannot capture the local effects of the data, while the Gaussian Process falls short when training data is scarce. The hybrid model combines the best of both worlds and performs well under all data scenarios.
• Figure 4: Audio Classification with spectrograms
• Figure 5: Iterative hybrid solver. The fixed point iteration alternates between discrete optimization problems (1) with solutions in the data-induced space (red triangles), and variational problems (2) with solutions in the Maxwell-conforming space (blue circles), the latter being accomplished by a modified finite element solver. [Adapted from kurz2022hybrid, Fig. 3]

Theorems & Definitions (1)

• Definition 1