Latent Refinement via Flow Matching for Training-free Linear Inverse Problem Solving

TL;DR

This work introduces LFlow, a training-free framework that solves linear inverse problems in a pretrained latent space by combining latent-flow priors with data-guided ODE sampling. It derives a principled, time-varying latent posterior covariance from Tweedie’s formula and the optimal vector field to produce covariance-aware guidance aligned with the generative dynamics. Empirically, LFlow attains state-of-the-art perceptual quality across Gaussian deblurring, motion deblurring, super-resolution, and inpainting, while remaining competitive in traditional pixel-based metrics and offering notable efficiency gains over prior latent-solvers. The approach is zero-shot with respect to the target task and highlights the importance of principled covariance modeling and latent-space dynamics for robust, high-fidelity inverse problem solving, with noted opportunities for runtime optimization and broader generalization in future work.

Abstract

Recent advances in inverse problem solving have increasingly adopted flow priors over diffusion models due to their ability to construct straight probability paths from noise to data, thereby enhancing efficiency in both training and inference. However, current flow-based inverse solvers face two primary limitations: (i) they operate directly in pixel space, which demands heavy computational resources for training and restricts scalability to high-resolution images, and (ii) they employ guidance strategies with prior-agnostic posterior covariances, which can weaken alignment with the generative trajectory and degrade posterior coverage. In this paper, we propose LFlow (Latent Refinement via Flows), a training-free framework for solving linear inverse problems via pretrained latent flow priors. LFlow leverages the efficiency of flow matching to perform ODE sampling in latent space along an optimal path. This latent formulation further allows us to introduce a theoretically grounded posterior covariance, derived from the optimal vector field, enabling effective flow guidance. Experimental results demonstrate that our proposed method outperforms state-of-the-art latent diffusion solvers in reconstruction quality across most tasks. The code will be publicly available at https://github.com/hosseinaskari-cs/LFlow .

Figures (18)

• Figure 1: (Top)LFlow pipeline: A VAE encoder maps the observation $\mathbf{y}$ to a latent $\mathbf{z}$, from which a noisy start $\mathbf{z}_{t_s}$ is formed. During VAE pre-training, the KL term encourages the encoder's approximate posterior $q_\phi(\mathbf{z}_0|\mathbf{x}_0)$ to approach the Gaussian prior $p_{\text{r}}(\mathbf{z}_0)=\mathcal{N}(\mathbf{0},\mathbf{I})$. A pretrained flow field $\mathbf{v}_{\boldsymbol{\theta}}(\mathbf{z}_t,t)$ defines the unconditional trajectory $\mathbf{z}_t$. At inference, orange arrows ($\boldsymbol{\rightarrow}$) denote likelihood-based guidance that corrects the prior field, yielding the conditional latent path $\mathbf{z}_t\!\mid\!\mathbf{y}$ toward $p(\mathbf{z}_0|\mathbf{y})$. Decoding with $\mathcal{D}_{\boldsymbol{\varphi}}$ produces $\hat{\mathbf{x}}_0$. (Bottom) Gaussian deblurring snapshots along the conditional path as $t$ decreases from $0.8$ to $0$.
• Figure 2: Posterior covariance values across $t \in [0, 0.9]$ for our method and OT-ODE, along with reconstructed images for a single FFHQ sample in the super-resolution task.
• Figure 3: Qualitative results on FFHQ test set. Row 1: Deblur (gaussian), Row 2: Deblur (motion), Row 3: SR$\times 4$, Row 4: Inpainting. Our approach better preserves fine image details than latent-based diffusion methods.
• Figure 3: Ablations on the effect of $\mathbb{C}\mathrm{ov}[\mathbf{z}_0 | \mathbf{z}_t]$. Bold indicates the best.
• Figure 4: Qualitative results on ImageNet test set. Row 1: Deblur (gaussian), Row 2: Deblur (motion), Row 3: SR$\times 4$, Row 4: Inpainting. Our method reconstructs fine image details more faithfully than the baselines.

Theorems & Definitions (27)

• Proposition 4.1: Posterior Sampling via Reverse-Time Conditional Flows
• Proposition 4.3: Bound on the Jacobian of the Vector Field
• Proposition 4.4: Optimal Vector Field
• Remark 4.5: Tightness of Bound
• Lemma A.1: Tweedie’s Mean Formula
• proof
• Lemma A.2: Tweedie’s Covariance Formula
• proof
• Lemma A.3: Connection between Posterior Mean and Vector Field
• proof