TAEN: A Model-Constrained Tikhonov Autoencoder Network for Forward and Inverse Problems

TL;DR

TAEN introduces a model-constrained Tikhonov autoencoder that enables forward and inverse PDE surrogate learning from a single arbitrary observation by employing data randomization as a regularizer. The framework spans naïve, model-constrained, and Tikhonov variants, with TAEN and TAEN-Full achieving near-Tikho-novik inverse performance while providing orders-of-magnitude speedups for forward solves. The authors provide forward and inverse error analyses for linear problems, demonstrate robust performance on 2D heat and Navier–Stokes problems, and advocate sequential over simultaneous training to improve convergence. Collectively, TAEN offers real-time capable, physics-enabled surrogate models that dramatically reduce data requirements and computational cost for PDE-associated forward/inverse tasks.

Abstract

Efficient real-time solvers for forward and inverse problems are essential in engineering and science applications. Machine learning surrogate models have emerged as promising alternatives to traditional methods, offering substantially reduced computational time. Nevertheless, these models typically demand extensive training datasets to achieve robust generalization across diverse scenarios. While physics-based approaches can partially mitigate this data dependency and ensure physics-interpretable solutions, addressing scarce data regimes remains a challenge. Both purely data-driven and physics-based machine learning approaches demonstrate severe overfitting issues when trained with insufficient data. We propose a novel Tikhonov autoencoder model-constrained framework, called TAE, capable of learning both forward and inverse surrogate models using a single arbitrary observation sample. We develop comprehensive theoretical foundations including forward and inverse inference error bounds for the proposed approach for linear cases. For comparative analysis, we derive equivalent formulations for pure data-driven and model-constrained approach counterparts. At the heart of our approach is a data randomization strategy, which functions as a generative mechanism for exploring the training data space, enabling effective training of both forward and inverse surrogate models from a single observation, while regularizing the learning process. We validate our approach through extensive numerical experiments on two challenging inverse problems: 2D heat conductivity inversion and initial condition reconstruction for time-dependent 2D Navier-Stokes equations. Results demonstrate that TAE achieves accuracy comparable to traditional Tikhonov solvers and numerical forward solvers for both inverse and forward problems, respectively, while delivering orders of magnitude computational speedups.

Figures (15)

• Figure 1: The schematic of TAEN approach. A sequential learning strategy is applied to learn the encoder and decoder in two phases. In Phase 1, at every epoch during training, we randomize the observation data with noise $\boldsymbol{\varepsilon} \sim \mathcal{N}\left( 0, \varepsilon^2 \left[ \boldsymbol{\mathop{\mathrm{diag}}\nolimits}\left( {\boldsymbol{y}} \right) \right]^2 \right)$ which is added to the observation data $\boldsymbol{y}$ to generate randomized observation samples. The randomized data is then fed into the encoder network $\Psi_{\mathrm{e}}$ to predict the inverse solution $\boldsymbol{u}^*$. The predicted inverse $\boldsymbol{u}^*$ is passed to the PtO map $B \circ \mathcal{G}$ to predict the observation data $B\boldsymbol{\omega}^{*}$. We minimize the encoder loss $\mathcal{L}_e$ for the optimal encoder network. In Phase 2, we randomize observations and pass through the pre-trained encoder network to produce inverse solutions $\boldsymbol{u}^*$. Then, $\boldsymbol{u}^*$ is treated as inputs to both the decoder network $\Psi_{\mathrm{d}}$ to produce $\boldsymbol{y}^*$ and PtO map to produce $B\boldsymbol{\omega}^*$. The decoder loss $\mathcal{L}_d$ is then minimized to find optimal decoder parameters.
• Figure 2: 2D heat equation.Left: Domain, boundary conditions, $16 \times 16$ finite element discretization mesh, and $10$ random observation locations. Middle: A sample of the PoI (the heat conductivity field). Right: The corresponding state (temperature field), observations (temperatures) are taken at 10 observed points. This pair of PoI and observation sample is used for training in one training sample case.
• Figure 3: 2D heat equation. Mean and standard deviation of absolute error for 500 test inverse solutions obtained from different approaches. Black points are observational locations. Note that TAEN and TAEN-Full (and similarly for nPOP and mcPOP approaches) have the same encoder (that encodes the inverse solutions), their (identical) results are shown on the 5th row. Relatively to the Tikhonov approach (Tik), the model-constrained approaches are more accurate, and within the model-constrained approaches, TAEN and TAEN-Full are the most accurate ones: in fact one training sample is sufficient for these two methods.
• Figure 4: 2D heat equation. The comparison of 500 test predicted forward solution (at the observational locations) obtained from different approaches. In all plots plot, the x-axis is the magnitude of the true observation, and the y-axis is the magnitude of the predicted observation, both axises has range of $\left[ 0,3 \right]$. The red line indicates the perfect matching between predictions and truth observations. Top row: Trained with $1$ training sample. Bottom row: Trained with $100$ training samples. As can be seen, model-constrained approaches are more accurate, and within the model-constrained approaches, TAEN and TAEN-Full are the most accurate ones: in fact one training sample is sufficient for these two methods.
• Figure 5: 2D heat equation. Mean and standard deviation of absolute pointwise error for 500 full state test solutions obtained from $\textcolor{black}{TAEN-Full}{}$ and mcOPO-Full. Black dots are the observational locations. The former is more accurate, especially for the case with one training sample in which it achieves two orders of magnitude smaller error.