Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

PocketDVDNet: Realtime Video Denoising for Real Camera Noise

Crispian Morris, Imogen Dexter, Fan Zhang, David R. Bull, Nantheera Anantrasirichai

TL;DR

The paper tackles real-time video denoising under realistic, multi-component sensor noise. It introduces PocketDVDNet, a lightweight denoiser produced via a sparsity-guided pruning and channel-reduction workflow, a physics-informed five-component noise model, and knowledge distillation from a retrained teacher on realistic noise. Key contributions include a 74% reduction in parameters with preserved or improved denoising quality, the removal of explicit noise-map inputs for the student, and successful processing of 5-frame temporal patches in real time. The results demonstrate practical impact for autofocus, autonomous systems, and surveillance, and the workflow offers a general framework for compressing temporal models for mobile deployment.

Abstract

Live video denoising under realistic, multi-component sensor noise remains challenging for applications such as autofocus, autonomous driving, and surveillance. We propose PocketDVDNet, a lightweight video denoiser developed using our model compression framework that combines sparsity-guided structured pruning, a physics-informed noise model, and knowledge distillation to achieve high-quality restoration with reduced resource demands. Starting from a reference model, we induce sparsity, apply targeted channel pruning, and retrain a teacher on realistic multi-component noise. The student network learns implicit noise handling, eliminating the need for explicit noise-map inputs. PocketDVDNet reduces the original model size by 74% while improving denoising quality and processing 5-frame patches in real-time. These results demonstrate that aggressive compression, combined with domain-adapted distillation, can reconcile performance and efficiency for practical, real-time video denoising.

PocketDVDNet: Realtime Video Denoising for Real Camera Noise

TL;DR

The paper tackles real-time video denoising under realistic, multi-component sensor noise. It introduces PocketDVDNet, a lightweight denoiser produced via a sparsity-guided pruning and channel-reduction workflow, a physics-informed five-component noise model, and knowledge distillation from a retrained teacher on realistic noise. Key contributions include a 74% reduction in parameters with preserved or improved denoising quality, the removal of explicit noise-map inputs for the student, and successful processing of 5-frame temporal patches in real time. The results demonstrate practical impact for autofocus, autonomous systems, and surveillance, and the workflow offers a general framework for compressing temporal models for mobile deployment.

Abstract

Live video denoising under realistic, multi-component sensor noise remains challenging for applications such as autofocus, autonomous driving, and surveillance. We propose PocketDVDNet, a lightweight video denoiser developed using our model compression framework that combines sparsity-guided structured pruning, a physics-informed noise model, and knowledge distillation to achieve high-quality restoration with reduced resource demands. Starting from a reference model, we induce sparsity, apply targeted channel pruning, and retrain a teacher on realistic multi-component noise. The student network learns implicit noise handling, eliminating the need for explicit noise-map inputs. PocketDVDNet reduces the original model size by 74% while improving denoising quality and processing 5-frame patches in real-time. These results demonstrate that aggressive compression, combined with domain-adapted distillation, can reconcile performance and efficiency for practical, real-time video denoising.
Paper Structure (12 sections, 5 equations, 4 figures, 2 tables)

This paper contains 12 sections, 5 equations, 4 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Proposed Method
  3. Model Pruning
  4. Architecture Modifications
  5. Multi-Component Noise Model
  6. Knowledge Distillation
  7. Results and Discussion
  8. Experiment settings
  9. Quantitative Comparison
  10. Qualitative Comparison
  11. Ablation Studies
  12. Conclusion

Figures (4)

  • Figure 1: Mean SSIM scores of all datasets from the models presented in \ref{['tbl:quantitative_comp']}. Our method performs extremely competitively given its size and runtime.
  • Figure 2: The proposed workflow consists of model compression via sparsity-guided pruning followed by performance recovery through knowledge distillation
  • Figure 3: PocketDVDNet processes 5-frame temporal patches through two cascaded denoising stages.
  • Figure 4: Qualitative examples demonstrating the superiority and shortcomings of our approach. Top: a sequence from the DAVIS test set, with synthetic noise. Bottom: a real-world example, with a 100 frame average as a pseudo GT. Zoom in for details.