Domain-adaptive Video Deblurring via Test-time Blurring

TL;DR

This work tackles the domain gap problem in dynamic-scene video deblurring by introducing a test-time domain adaptation framework that leverages a diffusion-based blurring model (ID-Blau) to synthesize domain-specific training pairs from blurred videos. Central to the approach are two modules: Relative Sharpness Detection (RSDM), which extracts pseudo-sharp patches from blurry frames, and Domain-adaptive Blur Condition Generation (DBCGM), which models motion-driven blur cues to create realistic blur conditions for the pseudo-sharp patches. These pseudo-sharp/blurred pairs are used to fine-tune state-of-the-art video deblurring models at test time, yielding substantial gains (up to 7.54 dB PSNR) across BSD, RealBlur, and RBVD datasets. The results demonstrate robust improvement in both quantitative metrics and qualitative visual fidelity, enabling more reliable deblurring under real-world domain shifts without requiring ground-truth sharp frames during inference.

Abstract

Dynamic scene video deblurring aims to remove undesirable blurry artifacts captured during the exposure process. Although previous video deblurring methods have achieved impressive results, they suffer from significant performance drops due to the domain gap between training and testing videos, especially for those captured in real-world scenarios. To address this issue, we propose a domain adaptation scheme based on a blurring model to achieve test-time fine-tuning for deblurring models in unseen domains. Since blurred and sharp pairs are unavailable for fine-tuning during inference, our scheme can generate domain-adaptive training pairs to calibrate a deblurring model for the target domain. First, a Relative Sharpness Detection Module is proposed to identify relatively sharp regions from the blurry input images and regard them as pseudo-sharp images. Next, we utilize a blurring model to produce blurred images based on the pseudo-sharp images extracted during testing. To synthesize blurred images in compliance with the target data distribution, we propose a Domain-adaptive Blur Condition Generation Module to create domain-specific blur conditions for the blurring model. Finally, the generated pseudo-sharp and blurred pairs are used to fine-tune a deblurring model for better performance. Extensive experimental results demonstrate that our approach can significantly improve state-of-the-art video deblurring methods, providing performance gains of up to 7.54dB on various real-world video deblurring datasets. The source code is available at https://github.com/Jin-Ting-He/DADeblur.

Figures (9)

• Figure 1: Illustration of the proposed domain adaptation method. It can generate domain-specific blur conditions for a blurring model to produce blurred video frames from the chosen pseudo-sharp patch. These blurred frames can be used to fine-tune video deblurring models and improve their performance in the target domain.
• Figure 2: Pipeline of the proposed domain adaptation scheme. Given a blurred video in the test domain, we use the Relative Sharpness Detection Module (RSDM) to extract relatively sharp patches to be pseudo-sharp patches and the Domain-adaptive Blur Condition Generation Module (DBCGM) to generate domain-specific blur conditions for blurring the pseudo-sharp patches using ID-Blau ID-Blau. Finally, the pseudo-sharp and blurred pairs are used to update a deblurring model for domain adaptation.
• Figure 3: The left figure shows the architecture of the Blur Magnitude Estimator (BME), which comprises a five-stage encoder-decoder design with Multi-Scale Feature Fusion (MSFF). The right figure is the architecture of MSFF. $E_{k}$ denotes the output of the encoder at stage $k$, and $\tilde{E}_{k}$ denotes its output after MSFF.
• Figure 4: Illustration of the Relative Sharpness Detection Module. We use the Blur Magnitude Estimator to obtain the blur magnitudes for a blurred image and crop a relatively sharp patch based on an adaptive sharpness threshold.
• Figure 5: Illustration of Domain-adaptive Blur Condition Generation Module. Given a pseudo-sharp patch and the collocated patches in its neighboring frames, we use the Blur Orientation Estimator to generate domain-specific blur orientations and the Blur Magnitude Estimator to generate domain-specific blur magnitudes. The blur magnitudes estimated from neighboring patches are used to modulate the pseudo-sharp patch by the Magnitude Adaptation Process. In the end, the domain-specific blur orientations and magnitudes are used for blurring.