Extreme Blind Image Restoration via Prompt-Conditioned Information Bottleneck

TL;DR

This work tackles Extreme Blind Image Restoration by reframing restoration as an Information Bottleneck problem and decomposing ELQ inputs into an intermediate, less-degraded LQ representation via a trainable projector $f_\theta$ before applying a frozen restoration model $g$. The central contribution is the Image Restoration Information Bottleneck (IRIB) objective, which couples an LQ reconstruction term with an HQ prior-matching term, optionally augmented by HQ fidelity terms. The method enables Look Forward Once for inference-time prompt refinement and offers a plug-and-play mechanism to strengthen existing BIR models without finetuning. Empirical results on severe degradation regimes demonstrate improved perceptual realism and structural fidelity, with clear benefits from LFO and the intermediate LQ space. The proposed framework is modular and broadly applicable to enhance fully blind restoration models under extreme degradations.

Abstract

Blind Image Restoration (BIR) methods have achieved remarkable success but falter when faced with Extreme Blind Image Restoration (EBIR), where inputs suffer from severe, compounded degradations beyond their training scope. Directly learning a mapping from extremely low-quality (ELQ) to high-quality (HQ) images is challenging due to the massive domain gap, often leading to unnatural artifacts and loss of detail. To address this, we propose a novel framework that decomposes the intractable ELQ-to-HQ restoration process. We first learn a projector that maps an ELQ image onto an intermediate, less-degraded LQ manifold. This intermediate image is then restored to HQ using a frozen, off-the-shelf BIR model. Our approach is grounded in information theory; we provide a novel perspective of image restoration as an Information Bottleneck problem and derive a theoretically-driven objective to train our projector. This loss function effectively stabilizes training by balancing a low-quality reconstruction term with a high-quality prior-matching term. Our framework enables Look Forward Once (LFO) for inference-time prompt refinement, and supports plug-and-play strengthening of existing image restoration models without need for finetuning. Extensive experiments under severe degradation regimes provide a thorough analysis of the effectiveness of our work.

Figures (11)

• Figure 1: Extreme blind image restoration with our Prompt-Conditioned Information Bottleneck method for various fully blind images. Our method produces high-quality results with fine detail.
• Figure 2: (a) Baseline Methods: Conventional restoration learns a single mapping ${\mathcal{X}}_\text{ELQ} \rightarrow {\mathcal{X}}_\text{HQ}$ from Extreme LQ images to HQ images. This is a highly ill-posed problem, thus training with only the HQ matching objective ${\mathcal{L}}_\text{HQ}$ becomes highly impractical. (b) Our Information Bottleneck Perspective introduces an intermediate distribution ${\mathcal{X}}_\text{LQ}$ and simulates a degrade-back channel, leading to an additional LQ reconstruction objective ${\mathcal{L}}_\text{LQ}$. (c) Look Forward Once (LFO): The decomposition of the ${\mathcal{X}}_\text{ELQ} \rightarrow {\mathcal{X}}_\text{HQ}$ problem to ${\mathcal{X}}_\text{ELQ} \rightarrow {\mathcal{X}}_\text{LQ} \rightarrow {\mathcal{X}}_\text{HQ}$ allows for auxiliary refinement in the intermediate LQ distribution during inference time. Prompt refinement via LFO improves performance of extreme image restoration by finding a finer image in the LQ domain.
• Figure 3: Overall Pipeline. Given an ELG image $x_\text{ELQ}$, the trainable projector $f_\theta$ produces $\hat{x}_\text{LQ}$; a frozen restoration model $g$ outputs $\hat{z}_\text{HQ}$. We simulate the degrade-back to obtain $\tilde{x}_\text{LQ}={\mathcal{D}}(z_\text{HQ})$ and minimize a blur-aware LQ reconstruction loss ${\mathcal{L}}_\text{LQ-recon}$. To regularize toward the HQ distribution, we apply ${\mathcal{L}}_\text{HQ-prior}$ and ${\mathcal{L}}_\text{HQ-fid}$. Gradients for the terms only flow through $f_\theta$ while $g$ and the prior remain frozen. Recurisve prompt refinement can condition $f_\theta / g$ during inference time, enabling Look Forward Once refinement.
• Figure 4: Qualitative Comparison.(c-d) One step restoration of OSEDiff and it's finetuned model; (e-g) Our methods and LFO prompt refinement variants. Although the fine-tuned OSEDiff improves image quality over the naïve baseline, it still produces occasional unrealistic artifacts. In contrast, the IRIB framework (ours) yields restorations that are both realistic and consistent with the ELQ input, effectively recovering the most probable image under severe degradations.
• Figure 5: LQ refinement with LFO.(a) The ELQ input. (b) From left to right: the initial LQ sample; the refined LQ sample after $1\times$ LFO; the refined LQ sample after $2\times$ LFO. Iterative refinement of the LQ sample with LFO forms coarse structure (e.g., eye) in LQ. (c) The final HQ output.