Diffeomorphic Template Registration for Atmospheric Turbulence Mitigation

TL;DR

This work tackles atmospheric turbulence mitigation by reframing the problem as diffeomorphic template registration without explicitly initializing a template. It selects a reference frame and models frame-to-template deformation as the aggregation of optical flows, justified by the Central Limit Theorem, then registers each frame to the latent template through a novel flow-inversion module. The latent irradiance is recovered via averaging warped frames and deblurred with a blind deconvolution step that treats the cumulative blur as a Gaussian PSF, enabling a simple, robust, and competitive end-to-end pipeline. The approach yields state-of-the-art performance on standard benchmarks, requires no training data, and provides a versatile, modular baseline that can be integrated into downstream turbulence mitigation pipelines with potential speedups from advanced optical-flow methods.

Abstract

We describe a method for recovering the irradiance underlying a collection of images corrupted by atmospheric turbulence. Since supervised data is often technically impossible to obtain, assumptions and biases have to be imposed to solve this inverse problem, and we choose to model them explicitly. Rather than initializing a latent irradiance ("template") by heuristics to estimate deformation, we select one of the images as a reference, and model the deformation in this image by the aggregation of the optical flow from it to other images, exploiting a prior imposed by Central Limit Theorem. Then with a novel flow inversion module, the model registers each image TO the template but WITHOUT the template, avoiding artifacts related to poor template initialization. To illustrate the robustness of the method, we simply (i) select the first frame as the reference and (ii) use the simplest optical flow to estimate the warpings, yet the improvement in registration is decisive in the final reconstruction, as we achieve state-of-the-art performance despite its simplicity. The method establishes a strong baseline that can be further improved by integrating it seamlessly into more sophisticated pipelines, or with domain-specific methods if so desired.

Figures (6)

• Figure 1: Schematic for Diffeomorphic Template Registration. To mitigate atmospheric turbulence, one needs a "reference template" to estimate distortions. Existing methods use simple heuristics (e.g. averaging) to initialize this reference template and are susceptible to blurry artifacts due to a lack of registration. Instead, we select a keyframe and calculate warping from this keyframe to other frames by optical flow. By the Central Limit Theorem, the average of these optical flows converges to the warping from the keyframe to the ground truth reference template. With a novel flow inversion algorithm (detailed in Sect. \ref{['sec:inversion']}), input frames can be registered to the reference template by Eq. \ref{['eq:template']}, even without explicit access to an initialized reference template. This registered reference template can seamlessly fit into existing downstream atmospheric turbulence mitigation pipelines, such as blind deconvolution and lucky image fusion.
• Figure 2: Sharper and more rigid reference template. Compared with naively averaging and space-time non-local averaging mao2020image, our method improves the quality of the reference template. Highlighted regions are zoomed-in and sharpened for the sake of visualization, better viewed in 3$\times$.
• Figure 3: Example of flow inversion and registration. Given the flow $w_{01}$, we map the non-integer endpoints to the four nearest pixels, then construct an inverse flow $w_{01}^{-1}$ by weighted averaging. When the corresponding weight is zero in $\alpha$, we fill in the missing values by inpainting. This scheme successfully registers $I_0$ to the target frame $I_1$, even without having access to the ground truth $I_1$.
• Figure 4: Approximating blurring kernels as Gaussians. Given a set of instantaneous wavefronts (visualized within a modulo of $2\pi$) and their PSFs, the per-frame PSF is approximated as a Gaussian function, characterized by shifts and a standard deviation.
• Figure 5: Sensitivity to the number of input frames. Left: Our method yields improvements over temporal averaging with respect to the number of frames. Right: Using 10 frames as input, ours outperforms NDIR in PSNR and SSIM.