HaineiFRDM: Explore Diffusion to Restore Defects in Fast-Movement Films

TL;DR

HaineiFRDM tackles the onerous task of restoring high-resolution fast-moving film footage by deploying a patch-based diffusion framework that runs on consumer GPUs. The method combines a patch-wise latent-space strategy with a Restoration-Guidance Network, a Global Frame Fusion pathway, and a Texture-Reconstruction Frequency Module, along with a patch-consistent inference scheme using global residuals to maintain frame-level coherence. It is trained and evaluated on a dataset that blends real-degraded and synthetically degraded footage, demonstrating superior restoration quality compared with open-source baselines on both synthetic and real data. The work offers a practical, scalable path toward high-fidelity film restoration in lab and industry settings, reducing manual effort while preserving authentic content texture and structure.

Abstract

Existing open-source film restoration methods show limited performance compared to commercial methods due to training with low-quality synthetic data and employing noisy optical flows. In addition, high-resolution films have not been explored by the open-source methods.We propose HaineiFRDM(Film Restoration Diffusion Model), a film restoration framework, to explore diffusion model's powerful content-understanding ability to help human expert better restore indistinguishable film defects.Specifically, we employ a patch-wise training and testing strategy to make restoring high-resolution films on one 24GB-VRAMR GPU possible and design a position-aware Global Prompt and Frame Fusion Modules.Also, we introduce a global-local frequency module to reconstruct consistent textures among different patches. Besides, we firstly restore a low-resolution result and use it as global residual to mitigate blocky artifacts caused by patching process.Furthermore, we construct a film restoration dataset that contains restored real-degraded films and realistic synthetic data.Comprehensive experimental results conclusively demonstrate the superiority of our model in defect restoration ability over existing open-source methods. Code and the dataset will be released.

Figures (6)

• Figure 1: Training pipeline of our proposed HaineiFRDM. The model is input degraded patched frames and extracts each frame features with Preprocess Module. Then the frame features input into ControlNet, in which we use Global-Prompt-Fusion Module and Global-Frame-Fusion Module to help the model have a global-frame awareness and use freuqency mdoule to maintain origin frame textures. Lastly, the learned ControlNet residuals is input into Unet to produce restored frame features $\hat{z_0}$ and we map $\hat{z_0}$ into RGB space and use defect loss to highlight defects region in restored frames.
• Figure 2: Synthesize data workflow.
• Figure 3: Example of a short caption, which should be centered.
• Figure 4: Example of a short caption, which should be centered.
• Figure 5: Example of a short caption, which should be centered.