FlowIE: Efficient Image Enhancement via Rectified Flow

TL;DR

FlowIE presents a fast, flow-based image enhancement framework that leverages the priors of a pre trained diffusion model through conditioned rectified flow. By learning a many-to-one transport from Gaussian noise to real images and guiding the path with a coarse restoration, FlowIE achieves high-quality results in fewer than five steps, substantially reducing inference time compared with diffusion sampling. The mean value sampling strategy further refines the transport path by selecting an optimal midpoint direction, improving detail and sharpness. The approach generalizes across tasks such as blind face restoration, blind SR, color enhancement, and inpainting, and extends to deraining and dehazing, demonstrating robust performance and practical efficiency for real-world deployment.

Abstract

Image enhancement holds extensive applications in real-world scenarios due to complex environments and limitations of imaging devices. Conventional methods are often constrained by their tailored models, resulting in diminished robustness when confronted with challenging degradation conditions. In response, we propose FlowIE, a simple yet highly effective flow-based image enhancement framework that estimates straight-line paths from an elementary distribution to high-quality images. Unlike previous diffusion-based methods that suffer from long-time inference, FlowIE constructs a linear many-to-one transport mapping via conditioned rectified flow. The rectification straightens the trajectories of probability transfer, accelerating inference by an order of magnitude. This design enables our FlowIE to fully exploit rich knowledge in the pre-trained diffusion model, rendering it well-suited for various real-world applications. Moreover, we devise a faster inference algorithm, inspired by Lagrange's Mean Value Theorem, harnessing midpoint tangent direction to optimize path estimation, ultimately yielding visually superior results. Thanks to these designs, our FlowIE adeptly manages a diverse range of enhancement tasks within a concise sequence of fewer than 5 steps. Our contributions are rigorously validated through comprehensive experiments on synthetic and real-world datasets, unveiling the compelling efficacy and efficiency of our proposed FlowIE. Code is available at https://github.com/EternalEvan/FlowIE.

Figures (18)

• Figure 1: The diagram of the proposed FlowIE. FlowIE leverages rectified flow to unleash the rich knowledge in the trained diffusion model and build straight-line paths between element distribution and clean images. The framework consistently achieves visually stunning results in a minimal number of steps and seamlessly generalizes to various image enhancement tasks, e.g., face inpainting, color enhancement and blind image super-resolution.
• Figure 2: As shown in (a), diffusion models ho2020denoisingrombach2022high solve ODEs in curve trajectories. Differently, rectified flow liu2022flow, illustrated in (b), bridges one-to-one straight paths between two distributions, thereby reducing the inference steps. (c) Our proposed FlowIE applies the flow-based framework to real-world data and discards the massive data preparation process. We construct a many-to-one mapping that predicts straight paths to clean images from arbitrary noise in an elementary distribution with proper guidance.
• Figure 3: The overall framework of FlowIE. FlowIE is a flow-based framework for image enhancement tasks. During training (left), we optimize the rectified flow ${\bm v}_\theta$ to bridge straight line paths ${\bm v}$ from an elementary distribution to clean images with proper guidance. We also developed the mean value sampling to improve the path estimation. During inference (right), we utilize the conditions from LQ to predict a linear direction toward the clean images on the midpoint of the transport curve, yielding high-quality and visually appealing results.
• Figure 4: Qualitative comparisons on CelebA-Test. FlowIE generates plausible HQ results with enough details and high identity similarity even though input faces are severely degraded, while previous methods produce visible artifacts or inconsistent faces.
• Figure 5: Qualitative comparisons on real-world faces. Our method performs plausible enhancement on real-world faces, producing high-fidelity and visually satisfactory faces. Compared to other methods, FlowIE enjoys robustness in front of challenging cases.