IQPFR: An Image Quality Prior for Blind Face Restoration and Beyond

TL;DR

IQPFR Tackles blind face restoration under unknown degradations with imperfect GT by introducing an Image Quality Prior (IQP) derived from no-reference IQA models. It fuses IQP with a discrete, dual-codebook prior and a quality-conditioned Transformer to steer restoration toward the highest feasible HQ outputs, while maintaining plug-and-play compatibility with existing BFR architectures. The method employs a common and HQ+ codebook, a quality-conditioned code prediction, and a discrete quality-optimization objective to mitigate over-optimization and adversarial biases. Empirical results across real and synthetic datasets show state-of-the-art perceptual quality and demonstrate the approach’s versatility when extended to related tasks like color and underwater image enhancement.

Abstract

Blind Face Restoration (BFR) addresses the challenge of reconstructing degraded low-quality (LQ) facial images into high-quality (HQ) outputs. Conventional approaches predominantly rely on learning feature representations from ground-truth (GT) data; however, inherent imperfections in GT datasets constrain restoration performance to the mean quality level of the training data, rather than attaining maximally attainable visual quality. To overcome this limitation, we propose a novel framework that incorporates an Image Quality Prior (IQP) derived from No-Reference Image Quality Assessment (NR-IQA) models to guide the restoration process toward optimal HQ reconstructions. Our methodology synergizes this IQP with a learned codebook prior through two critical innovations: (1) During codebook learning, we devise a dual-branch codebook architecture that disentangles feature extraction into universal structural components and HQ-specific attributes, ensuring comprehensive representation of both common and high-quality facial characteristics. (2) In the codebook lookup stage, we implement a quality-conditioned Transformer-based framework. NR-IQA-derived quality scores act as dynamic conditioning signals to steer restoration toward the highest feasible quality standard. This score-conditioned paradigm enables plug-and-play enhancement of existing BFR architectures without modifying the original structure. We also formulate a discrete representation-based quality optimization strategy that circumvents over-optimization artifacts prevalent in continuous latent space approaches. Extensive experiments demonstrate that our method outperforms state-of-the-art techniques across multiple benchmarks. Besides, our quality-conditioned framework demonstrates consistent performance improvements when integrated with prior BFR models. The code will be released.

Figures (17)

• Figure 1: Results on blind face restoration. DifFace+ is the DifFaceyue2022difface with our quality prior conditioned approach, having more details. Our IQPFR has the highest perceptual quality.
• Figure 2: Score distribution of FFHQ, where higher scores correspond to better quality. Notably, most ground-truth images only exhibit an average quality score between 0.7 and 0.8, with some residual degradation patterns. This limits the network’s ability to restore low-quality inputs to truly high-quality images, achieving scores closer to 0.9.
• Figure 3: Overall framework of IQPFR. (a) In the codebook learning stage, a dual-codebook architecture is proposed. The HQ+ codebook is learned to quantize $Z_h$ only when the quality score of $x_h$ is higher than the threshold $S_{thr}$. (b) In the codebook lookup stage, we input the quality score $S$ as a condition into Transformer T, which predicts two code sequences at the same time. The two codebooks are leveraged to look up the corresponding code entries. Finally, the NR-IQA model is utilized to calculate the quality loss $\mathcal{L}_{quality}$.
• Figure 4: Qualitative comparison on real-world datasets with different degrees of degradation: LFW (first row), WebPhoto-Test (second row), WIDER-Test (third row). Our results have better facial feature and more textures. Zoom in to see the details.
• Figure 5: Qualitative comparison on Celeba-Test. Our IQPFR demonstrates superior perceptual quality with aesthetic skin tone and enhanced detail preservation, notably in fine facial features like eyelashes.