Dropout Ensemble Kalman inversion for high dimensional inverse problems
Shuigen Liu, Sebastian Reich, Xin T. Tong
TL;DR
The paper introduces dropout ensemble Kalman inversion (DEKI) to overcome the subspace limitation of standard EKI in high-dimensional inverse problems. By employing dropout on ensemble deviations and a mean–deviation separated dynamics with adaptive steps and a truncated linearization, DEKI achieves exponential convergence under suitable conditions and scales linearly with the problem dimension, addressing computational bottlenecks in large-scale settings. A rigorous analysis establishes ensemble collapse control, Gauss–Newton–like convergence, and near-optimal zeroth-order query complexity; numerical tests on linear transport and Darcy’s law demonstrate robust performance with small ensembles where vanilla EKI fails. The approach offers a practical, scalable framework for derivative-free inversion with strong theoretical guarantees and broad applicability in high-dimensional PDE-constrained problems.
Abstract
Ensemble Kalman inversion (EKI) is an ensemble-based method to solve inverse problems. Its gradient-free formulation makes it an attractive tool for problems with involved formulation. However, EKI suffers from the ''subspace property'', i.e., the EKI solutions are confined in the subspace spanned by the initial ensemble. It implies that the ensemble size should be larger than the problem dimension to ensure EKI's convergence to the correct solution. Such scaling of ensemble size is impractical and prevents the use of EKI in high dimensional problems. To address this issue, we propose a novel approach using dropout regularization to mitigate the subspace problem. We prove that dropout-EKI converges in the small ensemble settings, and the computational cost of the algorithm scales linearly with dimension. We also show that dropout-EKI reaches the optimal query complexity, up to a constant factor. Numerical examples demonstrate the effectiveness of our approach.