Self-Supervised Selective-Guided Diffusion Model for Old-Photo Face Restoration

TL;DR

Self-Supervised Selective-Guided Diffusion (SSDiff), which leverages pseudo-reference faces generated by a pre-trained diffusion model under weak guidance, outperforms existing GAN-based and diffusion-based methods in perceptual quality, fidelity, and regional controllability.

Abstract

Old-photo face restoration poses significant challenges due to compounded degradations such as breakage, fading, and severe blur. Existing pre-trained diffusion-guided methods either rely on explicit degradation priors or global statistical guidance, which struggle with localized artifacts or face color. We propose Self-Supervised Selective-Guided Diffusion (SSDiff), which leverages pseudo-reference faces generated by a pre-trained diffusion model under weak guidance. These pseudo-labels exhibit structurally aligned contours and natural colors, enabling region-specific restoration via staged supervision: structural guidance applied throughout the denoising process and color refinement in later steps, aligned with the coarse-to-fine nature of diffusion. By incorporating face parsing maps and scratch masks, our method selectively restores breakage regions while avoiding identity mismatch. We further construct VintageFace, a 300-image benchmark of real old face photos with varying degradation levels. SSDiff outperforms existing GAN-based and diffusion-based methods in perceptual quality, fidelity, and regional controllability. Code link: https://github.com/PRIS-CV/SSDiff.

Figures (11)

• Figure 1: Existing face restoration methods face challenges with complex degradations in old face photos, which may result in residual breakage or unnatural facial colors. In contrast, our method restores sharp facial structures and natural facial color, and no visible breakage regions.
• Figure 2: (Left) LQ inputs provides supervision via a loss gradient scaled by $s$, aligning reverse diffusion outputs toward it. Strong guidance improves fidelity but harms perceptual quality, while weaker guidance enhances perceptual quality but reduces fidelity. (Right) Statistical analysis on our dataset shows that pseudo-labels generated under weak guidance ($s = 1{e^{-3}}$) preserve facial contours similar to LQ inputs, exhibit low similarity in semantic regions (e.g., eyes, mouth, nose), and achieve smooth breakage regions with color saturation distributions close to HQ faces.
• Figure 3: Overview of our method. Face parsing maps $\boldsymbol{P}$ and scratch masks $\boldsymbol{M}$ are estimated from input $\boldsymbol{y}_0$ using a pre-trained face parsing network yu2018bisenet and a pre-trained scratch detection network wan2020bringing. "Weak Guide" refers to the case shown in Fig. \ref{['fig: insight']} at $s = 1{e^{-3}}$, i.e., $s_w = 1{e^{-3}}$.
• Figure 4: In the early stage, gradient drops are minor, as updates focus on coarse structures like facial contours, which occupy a small region. In the later stage, gradients drop sharply due to refinements in face details, which cover most of a face.
• Figure 5: Qualitative comparisons of old-photo face images with existing methods.