Decoder-Free Distillation for Quantized Image Restoration

TL;DR

Quantization-aware Distilled Restoration (QDR), a framework for edge-deployed IR, and an Edge-Friendly Model (EFM) featuring a lightweight Learnable Degradation Gating (LDG) to dynamically modulate spatial degradation localization to stabilize the optimization tug-of-war.

Abstract

Quantization-Aware Training (QAT), combined with Knowledge Distillation (KD), holds immense promise for compressing models for edge deployment. However, joint optimization for precision-sensitive image restoration (IR) to recover visual quality from degraded images remains largely underexplored. Directly adapting QAT-KD to low-level vision reveals three critical bottlenecks: teacher-student capacity mismatch, spatial error amplification during decoder distillation, and an optimization "tug-of-war" between reconstruction and distillation losses caused by quantization noise. To tackle these, we introduce Quantization-aware Distilled Restoration (QDR), a framework for edge-deployed IR. QDR eliminates capacity mismatch via FP32 self-distillation and prevents error amplification through Decoder-Free Distillation (DFD), which corrects quantization errors strictly at the network bottleneck. To stabilize the optimization tug-of-war, we propose a Learnable Magnitude Reweighting (LMR) that dynamically balances competing gradients. Finally, we design an Edge-Friendly Model (EFM) featuring a lightweight Learnable Degradation Gating (LDG) to dynamically modulate spatial degradation localization. Extensive experiments across four IR tasks demonstrate that our Int8 model recovers 96.5% of FP32 performance, achieves 442 frames per second (FPS) on an NVIDIA Jetson Orin, and boosts downstream object detection by 16.3 mAP

Figures (20)

• Figure 1: Comparison of standard Distillation and our proposed Quantization-aware Distill Restoration (QDR) paradigm (Left). QDR enables high-quality restoration across diverse degradations (low-light, deraining, denoising, dehazing) (Middle). Our method recovers $\sim$96.5% of FP32 and significantly faster FPS (442) on the edge board (Right).
• Figure 2: Overview of the proposed QDR framework, which consists of two components: DFD and LMR. The DFD architecture ensures effective knowledge distillation under quantization, while the LMR module dynamically reweights the magnitudes of reconstruction and distillation losses to improve training stability and performance.
• Figure 3: Activation histograms at the bottleneck and decoder layers, illustrating the distribution differences between the FP32 and quantized models across different architectures.
• Figure 4: Architecture of our proposed Learnable Degradation Gating module containing two trainable scalers $[\alpha_{\ell}, \alpha_d]$ for controlling the fusion of Degradation importance map with residual features.
• Figure 5: Qualitative results on four restoration tasks: denoising, LLIE, deraining, and dehazing on full-precision settings.