DIPLI: Deep Image Prior Lucky Imaging for Blind Astronomical Image Restoration

TL;DR

DIPLI addresses the scarcity of astronomical training data by extending DIP to a multi-frame, back-projected reconstruction that uses TVNet-forced optical flow and Langevin-dynamics-based Monte Carlo averaging to mitigate overfitting. The method integrates a degradation model, neural priors, and variational inference to produce robust HQ reconstructions from 7–11 input frames, outperforming Lucky Imaging and several deep baselines on synthetic data and showing strong qualitative results on real data under domain shifts. Key contributions include the multi-frame back-projection framework, unsupervised optical-flow alignment, SGLD-based posterior sampling, and comprehensive benchmarks with ablations. This approach enables more reliable, data-efficient astronomical image restoration with practical applicability to real-world observations.

Abstract

Modern image restoration and super-resolution methods utilize deep learning due to its superior performance compared to traditional algorithms. However, deep learning typically requires large training datasets, which are rarely available in astrophotography. Deep Image Prior (DIP) bypasses this constraint by performing blind training on a single image. Although effective in some cases, DIP often suffers from overfitting, artifact generation, and instability. To overcome these issues and improve general performance, this work proposes DIPLI - a framework that shifts from single-frame to multi-frame training using the Back Projection technique, combined with optical flow estimation via the TVNet model, and replaces deterministic predictions with unbiased Monte Carlo estimation obtained through Langevin dynamics. A comprehensive evaluation compares the method against Lucky Imaging, a classical computer vision technique still widely used in astronomical image reconstruction, DIP, the transformer-based model RVRT, and the diffusion-based model DiffIR2VR-Zero. Experiments on synthetic datasets demonstrate consistent improvements, with the method outperforming baselines for SSIM, PSNR, LPIPS, and DISTS metrics in the majority of cases. In addition to superior reconstruction quality, the model also requires far fewer input images than Lucky Imaging and is less prone to overfitting or artifact generation. Evaluation on real-world astronomical data, where domain shifts typically hinder generalization, shows that the method maintains high reconstruction quality, confirming practical robustness.

Figures (7)

• Figure 1: Generalized UNNP reconstruction framework. Before starting optimization, the sampler generates a fixed input signal $z$ for the generator network $G_{\theta}$. $G_\theta$ learns to reconstruct the high-quality image $y^*$ based on the implicit regularization prior given by the network architecture. After a predefined forward degradation model $f$, the reconstruction $y^*$ is compared to a given set of LQ observations using the loss function $\mathcal{L}$ and additional information (such as optical flows, PSF, etc.) $\omega$.
• Figure 2: Deep Image Prior optimization. The goal of DIP optimization is to find parameters $\theta$ such that for a predetermined noise $z$, the output of the generator network $G_{\theta}$ will be a high-quality image $y^*$. The network $G_{\theta}$ is trained by minimizing the task-specific loss function $\mathcal{L}$ (usually mean squared error) between the given observation $x$ and the reconstruction $y^*$ distorted with the degradation process $f$.
• Figure 3: Comparison of DIPLI reconstruction quality with different number of received LQ frames. While multiple frames provide complementary information, excessive inputs cause the model to fit inter-frame variations, reducing precision. Optimal performance occurs with approximately 7–13 frames.
• Figure 4: Comparison of several optical flow computation methods for a pair of images. The basis image PIVOT is depicted on the top left and followed by several maps of pixel-wise error between the pivot image and the compensated observation obtained with the corresponding methods.
• Figure 5: Comparison of DIPLI reconstruction quality with different values of the SGLD strength coefficient$\sigma_{\xi}$. Following the approach of Cheng_2019, the results indicate a tradeoff between noise impedance and the number of iterations required for convergence. Higher $\sigma_{\xi}$ values increase the convergence time. For astronomical images in this dataset, the optimal setting was $\sigma_{\xi} = 0.0025$ with 6500 iterations.