DELNet: Continuous All-in-One Weather Removal via Dynamic Expert Library

TL;DR

The paper tackles the problem of continual learning for all-weather image restoration, where models must adapt to unseen weather degradations without retraining. It introduces DELNet, a framework that uses a Judging Valve to estimate task similarity and a Dynamic Expert Library to assemble a small, task-specific set of adapters via Top-K routing, all built upon a Deep Feature Enhancement backbone. A multi-level loss, including $L_{sw}$, $L_{kd}$, $L_p$, $L_{reg}$, and $L_{div}$, is used to promote knowledge transfer while preventing forgetting, formalized as $L_{total} = L_{sw} + L_{kd} + L_p + L_{reg} + L_{div}$. Empirical results on RESIDE, Rain100H, and Snow100K show consistent improvements over continual baselines and competitive performance with static all-in-one models, while greatly reducing retraining costs and enabling practical deployment in dynamic environments.

Abstract

All-in-one weather image restoration methods are valuable in practice but depend on pre-collected data and require retraining for unseen degradations, leading to high cost. We propose DELNet, a continual learning framework for weather image restoration. DELNet integrates a judging valve that measures task similarity to distinguish new from known tasks, and a dynamic expert library that stores experts trained on different degradations. For new tasks, the valve selects top-k experts for knowledge transfer while adding new experts to capture task-specific features; for known tasks, the corresponding experts are directly reused. This design enables continuous optimization without retraining existing models. Experiments on OTS, Rain100H, and Snow100K demonstrate that DELNet surpasses state-of-the-art continual learning methods, achieving PSNR gains of 16\%, 11\%, and 12\%, respectively. These results highlight the effectiveness, robustness, and efficiency of DELNet, which reduces retraining cost and enables practical deployment in real-world scenarios.

Figures (6)

• Figure 1: Comparison chart of methods for eliminating adverse weather conditions. Including single removal method, unified method, all-in-one continuous learning method and our method.
• Figure 2: Overview of DELNet. The framework integrates a Deep Feature Enhancement (DFE) network, a judging valve, and a dynamic expert library for continual multi-weather image restoration.
• Figure 3: Three different adapter and expert modes: (a) All trainable, (b) Blending modes, and (c) All frozen.
• Figure 4: Visualization of image restoration results on RESIDE, Rain100H, and Snow100K datasets.
• Figure 5: Ablation on expert number: (a) Impact of the number of experts (b) Ablation on expert number (Haze).