All-in-One Video Restoration under Smoothly Evolving Unknown Weather Degradations

TL;DR

This paper introduces the Smoothly Evolving Unknown Degradations (SEUD) scenario, where both the active degradation set and degradation intensity change continuously over time, and proposes an all-in-One Recurrent Conditional and Adaptive prompting Network (ORCANet).

Abstract

All-in-one image restoration aims to recover clean images from diverse unknown degradations using a single model. But extending this task to videos faces unique challenges. Existing approaches primarily focus on frame-wise degradation variation, overlooking the temporal continuity that naturally exists in real-world degradation processes. In practice, degradation types and intensities evolve smoothly over time, and multiple degradations may coexist or transition gradually. In this paper, we introduce the Smoothly Evolving Unknown Degradations (SEUD) scenario, where both the active degradation set and degradation intensity change continuously over time. To support this scenario, we design a flexible synthesis pipeline that generates temporally coherent videos with single, compound, and evolving degradations. To address the challenges in the SEUD scenario, we propose an all-in-One Recurrent Conditional and Adaptive prompting Network (ORCANet). First, a Coarse Intensity Estimation Dehazing (CIED) module estimates haze intensity using physical priors and provides coarse dehazed features as initialization. Second, a Flow Prompt Generation (FPG) module extracts degradation features. FPG generates both static prompts that capture segment-level degradation types and dynamic prompts that adapt to frame-level intensity variations. Furthermore, a label-aware supervision mechanism improves the discriminability of static prompt representations under different degradations. Extensive experiments show that ORCANet achieves superior restoration quality, temporal consistency, and robustness over image and video-based baselines. Code is available at https://github.com/Friskknight/ORCANet-SEUD.

Figures (11)

• Figure 1: Examples of SEUD scenario. (a) is a real-world SEUD scenario captured by smartphone. (b) and (c) are synthetically generated sequences. In (c), the set of present degradation types changes over time in addition to intensity variation. The numerical values below each frame represent the estimated normalized degradation intensities used in synthesis process.
• Figure 2: Comparison of recovery results in SEUD scenarios. Existing restoration methods show artifacts and instability under SEUD, while our approach maintains more consistent recovery.
• Figure 3: SEUD Weather Synthesis Pipeline. Left: The input consists of clear video frames and their corresponding depth maps. Middle: Each particle is assigned a random depth and a set of physical attributes which correlate with depth. Distant particles are thinner, shorter, slower, and more transparent, and are more likely to be occluded by the foreground visibility mask. Their motion follows gravity $\mathbf{g}$ and the time-varying wind field $\mathbf{w}(t)$. Right: Example intensity functions $\beta(t)$ and $f(t)$ control the temporal evolution of haze and precipitation, producing synthetic videos with continuous, mixed, and time-varying degradations.
• Figure 4: OCRANet Framework Overview. The network adopts a bidirectional recurrent propagation design for video restoration. Each frame is first processed by the CIED module for depth-guided coarse dehazing and intensity estimation. Temporal features are refined through multiple ORCA Units (ORCAU) with flow-aligned prompts generated by the FPG module. A pixel-shuffle decoder reconstructs clean frames. Detailed module structures of CIED, ORCAU, and FPG are shown on the right (a)-(c).
• Figure 5: Recovery results of single-type degraded videos in setting 1.