A Test-Time Learning Approach to Reparameterize the Geophysical Inverse Problem with a Convolutional Neural Network

TL;DR

This work addresses ill-posed DC resistivity inversions by testing whether implicit regularization from CNNs can aid inversion when weights are learned at test time (TTL). The authors propose DIP-Inv, which reparameterizes the subsurface model as $ m = L_w(z) $ and optimizes CNN weights $ w $ to minimize $ \\mathcal{L} = (1-\\beta) \\phi_d + \\beta \\phi_m $, with $ \\phi_m = \\\|L_w(z) - m_{ref}\\|^1 $. Through synthetic DC resistivity experiments, DIP-Inv matches conventional data fits while sometimes delivering superior dip recovery and better separation of close anomalies; the bi-linear upsampling in the CNN and dropout can substantially contribute to smoothing effects and regularization. The study also details practical integration of SimPEG forward modeling with PyTorch for TTL optimization and identifies timing and architectural considerations. Overall, the implicit regularization from CNNs offers a viable, training-data-free priors-based alternative for geophysical inversions and can be extended to other Tikhonov-style problems.

Abstract

Regularization is critical for solving ill-posed geophysical inverse problems. Explicit regularization is often used, but there are opportunities to explore the implicit regularization effects that are inherent in a Neural Network structure. Researchers have discovered that the Convolutional Neural Network (CNN) architecture inherently enforces a regularization that is advantageous for addressing diverse inverse problems in computer vision, including de-noising and in-painting. In this study, we examine the applicability of this implicit regularization to geophysical inversions. The CNN maps an arbitrary vector to the model space. The predicted subsurface model is then fed into a forward numerical simulation to generate corresponding predicted measurements. Subsequently, the objective function value is computed by comparing these predicted measurements with the observed measurements. The backpropagation algorithm is employed to update the trainable parameters of the CNN during the inversion. Note that the CNN in our proposed method does not require training before the inversion, rather, the CNN weights are estimated in the inversion process, hence this is a test-time learning (TTL) approach. In this study, we choose to focus on the Direct Current (DC) resistivity inverse problem, which is representative of typical Tikhonov-style geophysical inversions (e.g. gravity, electromagnetic, etc.), to test our hypothesis. The experimental results demonstrate that the implicit regularization can be useful in some DC resistivity inversions. We also provide a discussion of the potential sources of this implicit regularization introduced from the CNN architecture and discuss some practical guides for applying the proposed method to other geophysical methods.

Figures (16)

• Figure 1: (a) is the true conductivity model. (b) is the result of a $l_2$ inversion using only smallness. (c) is the result of a $l_2$ inversion using smallness and smoothness. (d) is the result of a $l_1$ inversion using smallness and smoothness. (e) is the result of a $l_1$ inversion using smallness, smoothness, and sensitivity weighting.
• Figure 2: Proposed inversion pipeline. DIP-Inv reparameterizes the mesh space into the CNN-weights space.
• Figure 3: Architecture of DIP-Inv. The structure of hidden blocks 2-4 is shown on the right (hb denotes hidden block). More details are shown in Table. \ref{['table:1']}.
• Figure 4: (a) is the true conductivity model for Case 1. The top layer has a conductivity of 0.02 S/m. The dike, with a dip angle of 45 degrees, has a conductivity of 0.1 S/m. The background has a conductivity of 0.01 S/m. (b) shows the electrical potential and current density with the 6th transmitter for Case 1. (c) shows the pseudosection for Case 1.
• Figure 5: (a) is the true conductivity model for Case 2. The cylinder and the dike have a conductivity of 0.1 S/m. The background has a conductivity of 0.01 S/m. (b) shows the electrical potential and current density with the 14th transmitter for Case 2. (c) shows the pseudosection for Case 2.