LaverNet: Lightweight All-in-one Video Restoration via Selective Propagation

TL;DR

The paper introduces LaverNet, a 362K-parameter lightweight all-in-one video restoration network designed to handle time-varying degradations. It employs a Selective Propagation Module (SPM) to transmit degradation-agnostic temporal features and a Lightweight Enhancement Module (LEM) to per-frame address diverse degradations, achieving robust temporal modeling with minimal parameters. Across DAVIS and Set8 benchmarks, LaverNet matches or surpasses larger models while dramatically reducing parameter count and runtime, demonstrating that targeted propagation and lightweight design can enable practical all-in-one video restoration. This work highlights the feasibility of efficient, degradation-aware video restoration suitable for real-world, multi-degradation scenarios.

Abstract

Recent studies have explored all-in-one video restoration, which handles multiple degradations with a unified model. However, these approaches still face two challenges when dealing with time-varying degradations. First, the degradation can dominate temporal modeling, confusing the model to focus on artifacts rather than the video content. Second, current methods typically rely on large models to handle all-in-one restoration, concealing those underlying difficulties. To address these challenges, we propose a lightweight all-in-one video restoration network, LaverNet, with only 362K parameters. To mitigate the impact of degradations on temporal modeling, we introduce a novel propagation mechanism that selectively transmits only degradation-agnostic features across frames. Through LaverNet, we demonstrate that strong all-in-one restoration can be achieved with a compact network. Despite its small size, less than 1\% of the parameters of existing models, LaverNet achieves comparable, even superior performance across benchmarks.

Figures (4)

• Figure 1: Visualization of (a) the frame features and (b) the selected features using our proposed Selective Propagation Mechanism (SPM). It can be observed that SPM selectively propagates degradation-agnostic features, facilitating more efficient propagation of temporally correlated features.
• Figure 2: Architecture overview. (a) Overall architecture of LaverNet, comprising two key modules: (b) Selective Propagation Module (SPM), which propagates degradation-agnostic temporal information, and (c) Lightweight Enhancement Module (LEM), which efficiently handles each type of degradation in every frame.
• Figure 3: Qualitative results on the "tennis-vest" video from DAVIS-test ($t=12$), from which one could observe that existing methods tend to produce blurry outputs with visible artifacts. In contrast, our method effectively restores the sharp details of the wire mesh, yielding results that are closer to the ground truth.
• Figure 4: Qualitative results on the "helicopter" video from DAVIS-test in the noise&compression degradation combination, from which one could observe that existing produce blurry or noisy results, while our method restores clearer outlines that are closer to GT.