Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

FMPlug: Plug-In Foundation Flow-Matching Priors for Inverse Problems

Yuxiang Wan, Ryan Devera, Wenjie Zhang, Ju Sun

TL;DR

This paper tackles ill-posed inverse problems by leveraging domain-agnostic foundation flow-matching priors. It introduces FMPlug, a plug-in framework that strengthens foundation priors through two key ideas: a time-dependent warm-up that uses a learnable starting point aligned with the observed data, and a sharp Gaussianity regularization realized by latent-space sphere projection. Empirically, FMPlug—especially its warm-up plus regularization variant—outperforms state-of-the-art foundation-prior methods on image restoration tasks such as super-resolution and Gaussian deblurring, demonstrating improved data fidelity and perceptual quality. The approach enables robust, domain-agnostic inverse problem solving with foundation FM priors, reducing reliance on domain-specific models and facilitating broader applicability.

Abstract

We present FMPlug, a novel plug-in framework that enhances foundation flow-matching (FM) priors for solving ill-posed inverse problems. Unlike traditional approaches that rely on domain-specific or untrained priors, FMPlug smartly leverages two simple but powerful insights: the similarity between observed and desired objects and the Gaussianity of generative flows. By introducing a time-adaptive warm-up strategy and sharp Gaussianity regularization, FMPlug unlocks the true potential of domain-agnostic foundation models. Our method beats state-of-the-art methods that use foundation FM priors by significant margins, on image super-resolution and Gaussian deblurring.

FMPlug: Plug-In Foundation Flow-Matching Priors for Inverse Problems

TL;DR

This paper tackles ill-posed inverse problems by leveraging domain-agnostic foundation flow-matching priors. It introduces FMPlug, a plug-in framework that strengthens foundation priors through two key ideas: a time-dependent warm-up that uses a learnable starting point aligned with the observed data, and a sharp Gaussianity regularization realized by latent-space sphere projection. Empirically, FMPlug—especially its warm-up plus regularization variant—outperforms state-of-the-art foundation-prior methods on image restoration tasks such as super-resolution and Gaussian deblurring, demonstrating improved data fidelity and perceptual quality. The approach enables robust, domain-agnostic inverse problem solving with foundation FM priors, reducing reliance on domain-specific models and facilitating broader applicability.

Abstract

We present FMPlug, a novel plug-in framework that enhances foundation flow-matching (FM) priors for solving ill-posed inverse problems. Unlike traditional approaches that rely on domain-specific or untrained priors, FMPlug smartly leverages two simple but powerful insights: the similarity between observed and desired objects and the Gaussianity of generative flows. By introducing a time-adaptive warm-up strategy and sharp Gaussianity regularization, FMPlug unlocks the true potential of domain-agnostic foundation models. Our method beats state-of-the-art methods that use foundation FM priors by significant margins, on image super-resolution and Gaussian deblurring.

Paper Structure

This paper contains 9 sections, 12 equations, 5 figures, 3 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Technical background & related Work
  3. Flow matching (FM)
  4. Pretrained FM priors for IPs
  5. Foundation FM priors for IPs
  6. Method
  7. Time-dependent warm-up strategy
  8. Gaussianity regularization
  9. Experiment

Figures (5)

  • Figure 1: Plot of the function $h(\boldsymbol z_0)$ (after a change of variable $u = \norm{\boldsymbol z_0}_2^2$). An ideal regularization function should blow up sharply away from the narrow concentration region in orange to promote Gaussianity effectively.
  • Figure 2: Qualitative results for $4 \times$ super-resolution on the AFHQ dataset.
  • Figure 3: Qualitative results for $4 \times$ super-resolution on the DIV2K dataset.
  • Figure 4: Qualitative results for Gaussian deblurring on the AFHQ dataset.
  • Figure 5: Qualitative results for Gaussian deblurring on DIV2K.