Towards Unified Image Deblurring using a Mixture-of-Experts Decoder

TL;DR

This paper tackles the challenge of deblurring under diverse blur types by proposing DeMoE, a single all-in-one model that uses a mixture-of-experts (MoE) decoder to route features based on detected blur degradations. Built on a degraded-aware pre-training paradigm, the method leverages a degradation classifier at the encoder and a soft-gating MoE in the decoder to jointly restore images across global/local motion, low-light, and defocus blur. Empirical results on the newly introduced AIO-Blur dataset show DeMoE achieving competitive PSNR/SSIM with lower computational cost than task-specific or other all-in-one methods, and robustness can be enhanced by manual expert selection in out-of-distribution scenarios. The work offers practical implications for real-world imaging pipelines by reducing the need for multiple specialized models while maintaining restoration quality and flexibility, and will release code publicly.

Abstract

Image deblurring, removing blurring artifacts from images, is a fundamental task in computational photography and low-level computer vision. Existing approaches focus on specialized solutions tailored to particular blur types, thus, these solutions lack generalization. This limitation in current methods implies requiring multiple models to cover several blur types, which is not practical in many real scenarios. In this paper, we introduce the first all-in-one deblurring method capable of efficiently restoring images affected by diverse blur degradations, including global motion, local motion, blur in low-light conditions, and defocus blur. We propose a mixture-of-experts (MoE) decoding module, which dynamically routes image features based on the recognized blur degradation, enabling precise and efficient restoration in an end-to-end manner. Our unified approach not only achieves performance comparable to dedicated task-specific models, but also shows promising generalization to unseen blur scenarios, particularly when leveraging appropriate expert selection. Code available at https://github.com/cidautai/DeMoE.

Figures (14)

• Figure 1: Illustration of different types of blur restoration. At top, a set of possible deblurring restoration strategies. At bottom, a comparison of task-specific methods and our proposed DeMoE. Previous approaches can tackle only specific types of blur and fail to deal with different types of blur. Our all-in-one deblurring method addresses multiple types of blur degradations using a single model. Zoom in for optimal comparison.
• Figure 2: Network Similarity Study across Deblurring tasks. We show the layer-wise correlations between different deblurring versions of NAFNet i.e., each bar represents the weights correlation of one layer. Excluding the LOLBlur dataset, all the other weights hold a high-correlation (greater than 0.7 Asuero2006, over the red line). For instance, this reveals that a model trained to solve defocus (DPDD) learns similar representations as a model trained for motion deblurring (GOPRO).
• Figure 3: DeMoE Network Architecture. We adopt NAFNet as backbone. The encoder generates a feature space optimized for both restoration and degradation classification. The router uses the encoded features to determine the degradation and assigns weights to each expert. During training, all experts contribute to restoration; during inference, only the top-k experts—selected based on router weights—are used to produce the final output.
• Figure 4: (Left) t-SNE distribution of the testing images in the AIO-Blur and AIO-Blur-OOD datasets. (Right) Main specifications of each dataset used for training and testing. We use three real-world datasets to test the robustness of the models in out of distribution (OOD) scenarios.
• Figure 5: Qualitative comparison of the general deblur methods. Methods with $^{*}$ are SOTA task-specific methods. Results, from top to bottom, of the following datasets: DPDD, LOLBlur, ReLoBlur, GoPro and RealBlur. Our method, DeMoE, provides results comparable to SOTA.