SFP: Real-World Scene Recovery Using Spatial and Frequency Priors

TL;DR

This work tackles real-world scene recovery under diverse degradations without relying on large paired datasets. It introduces Spatial and Frequency Priors (SFP), which combines a spatial-domain transmission estimation via spectral projection with two statistical priors in the frequency domain, followed by a fusion of restored, enhanced, and input components. The method demonstrates superior performance across multiple real-world tasks (haze, nighttime haze, underwater, sandstorm, remote sensing) and includes extensive ablations, underscoring the contribution of each component. Overall, SFP offers a lightweight, generalizable framework for robust scene restoration by exploiting complementary priors in spatial and frequency domains.

Abstract

Scene recovery serves as a critical task for various computer vision applications. Existing methods typically rely on a single prior, which is inherently insufficient to handle multiple degradations, or employ complex network architectures trained on synthetic data, which suffer from poor generalization for diverse real-world scenarios. In this paper, we propose Spatial and Frequency Priors (SFP) for real-world scene recovery. In the spatial domain, we observe that the inverse of the degraded image exhibits a projection along its spectral direction that resembles the scene transmission. Leveraging this spatial prior, the transmission map is estimated to recover the scene from scattering degradation. In the frequency domain, a mask is constructed for adaptive frequency enhancement, with two parameters estimated using our proposed novel priors. Specifically, one prior assumes that the mean intensity of the degraded image's direct current (DC) components across three channels in the frequency domain closely approximates that of each channel in the clear image. The second prior is based on the observation that, for clear images, the magnitude of low radial frequencies below 0.001 constitutes approximately 1% of the total spectrum. Finally, we design a weighted fusion strategy to integrate spatial-domain restoration, frequency-domain enhancement, and salient features from the input image, yielding the final recovered result. Extensive evaluations demonstrate the effectiveness and superiority of our proposed SFP for scene recovery under various degradation conditions.

Figures (13)

• Figure 1: Examples of scene recovery under different degradation conditions, including daytime/nighttime haze, sandstorm, remote sensing, and underwater scenarios.
• Figure 2: Illustration of our spatial-domain prior. (a) Schematic diagram of approximating the transmission map $t(x)$ by projecting $1-\mathcal{I}(x)$ onto the spectral direction $\mathcal{S}(x)$. (b) Comparison of MSE deviations between estimated and ground-truth transmissions for various priors on the Haze4K test set.
• Figure 3: The projection of the inverted degraded image $1-\mathcal{I}(x)$ onto the spectral direction $\mathcal{S}(x)$ closely resembles the ground-truth scene transmission. (a) Degraded images. (b) Transmission maps estimated by our spatial-domain prior. (c) Ground-truth transmission maps.
• Figure 4: Statistical distributions of the absolute differences between the normalized DC component (DC$\in \left[ {0,1} \right]$) of each channel (e.g. $\mathcal{R}$, $\mathcal{G}$, $\mathcal{B}$) in clear images and the mean DC value across the three channels of their corresponding degraded images, computed on 1000 degraded-clean image pairs. The yellow dashed curves (cumulative distributions) show that over 80% of image pairs have absolute differences below 0.2 in all three channels.
• Figure 5: Statistical distributions of the percentages of low radial frequency components ($<$0.001) in the $\mathcal{R}$, $\mathcal{G}$, and $\mathcal{B}$ channels for 1000 clear and 1000 degraded images. The horizontal axis represents the percentage of spectral magnitudes ($<$0.001) in the total spectral magnitude, while the vertical axis indicates the number of images. The red dashed lines denote the mean values for each channel. The bottom row presents examples of clear and degraded images along with their corresponding low radial frequency percentages ($<$0.001).