RestorerID: Towards Tuning-Free Face Restoration with ID Preservation

TL;DR

Experimental results on the Celeb-Ref dataset and real-world scenarios demonstrate that RestorerID effectively delivers high-quality face restoration with ID preservation, achieving a superior performance compared to the test-tuning approaches and other reference-guided ones.

Abstract

Blind face restoration has made great progress in producing high-quality and lifelike images. Yet it remains challenging to preserve the ID information especially when the degradation is heavy. Current reference-guided face restoration approaches either require face alignment or personalized test-tuning, which are unfaithful or time-consuming. In this paper, we propose a tuning-free method named RestorerID that incorporates ID preservation during face restoration. RestorerID is a diffusion model-based method that restores low-quality images with varying levels of degradation by using a single reference image. To achieve this, we propose a unified framework to combine the ID injection with the base blind face restoration model. In addition, we design a novel Face ID Rebalancing Adapter (FIR-Adapter) to tackle the problems of content unconsistency and contours misalignment that are caused by information conflicts between the low-quality input and reference image. Furthermore, by employing an Adaptive ID-Scale Adjusting strategy, RestorerID can produce superior restored images across various levels of degradation. Experimental results on the Celeb-Ref dataset and real-world scenarios demonstrate that RestorerID effectively delivers high-quality face restoration with ID preservation, achieving a superior performance compared to the test-tuning approaches and other reference-guided ones. The code of RestorerID is available at \url{https://github.com/YingJiacheng/RestorerID}.

Figures (10)

• Figure 1: As image degradation increases, the blind face restoration approach CodeFormer CodeFormer2022 can restore images but fails to preserve ID consistency (see the second row). In contrast, Our RestorerID, incorporating reference ID priors, generates restored images with consistent ID information (see the third row).
• Figure 2: (a) Our RestorerID framework integrates LQ structural information and reference ID information into a unified diffusion UNet. RestorerID adopts the FIR-Adapter and Adaptive ID-Scale Adjusting module to balance the above two types of information. (b) The FIR-Adapter effectively fuses the LQ structure conditions with reference ID embeddings through an adaptive training mechanism. (c) The Adaptive ID-Scale Adjusting module adjusts the ID injection degree based on degradation assessment.
• Figure 3: The outputs produced by the base model, base model + ID injection, and Our method. Blue, red, and green boxes highlight the contours misalignment, pose inconsistency and content mistakes, respectively.
• Figure 4: The restored images under light and heavy degradation using increasing ID-Scale values and our adaptive adjusting strategy. Red boxes highlight the facial details. Please zoom in for the best view.
• Figure 5: The curves of the average ID similarity values with respect to IP-Scale across different MUSIQ intervals.