Combining Generative and Geometry Priors for Wide-Angle Portrait Correction

TL;DR

This work proposes encapsulating the generative face prior as a guided natural manifold to facilitate the correction of facial regions, excelling not only in quantitative measures such as line straightness and shape consistency metrics but also in terms of perceptual visual quality.

Abstract

Wide-angle lens distortion in portrait photography presents a significant challenge for capturing photo-realistic and aesthetically pleasing images. Such distortions are especially noticeable in facial regions. In this work, we propose encapsulating the generative face prior as a guided natural manifold to facilitate the correction of facial regions. Moreover, a notable central symmetry relationship exists in the non-face background, yet it has not been explored in the correction process. This geometry prior motivates us to introduce a novel constraint to explicitly enforce symmetry throughout the correction process, thereby contributing to a more visually appealing and natural correction in the non-face region. Experiments demonstrate that our approach outperforms previous methods by a large margin, excelling not only in quantitative measures such as line straightness and shape consistency metrics but also in terms of perceptual visual quality. All the code and models are available at https://github.com/Dev-Mrha/DualPriorsCorrection.

Figures (9)

• Figure 1: Example of distorted and corrected images in comparison to other methods. Thanks to the proposed generative and geometry priors, our results have straighter lines in the background and more natural-looking faces compared to other methods.
• Figure 2: Overview of our framework. It contains three parts: LineCNet for the geometric distortions in the background, FaceCNet for correcting the face region using the generative face prior, and a post-process step for fusing the corrected face region into the background.
• Figure 3: Examples of our face correction w/ and w/o using generative prior. Through the results of correcting flow, it can be observed that the correction is concentrated at the position where the StyleGAN face structure deviates from the input.
• Figure 4: Comparison results of background correction with others. We can observe that the proposed LineCNet can better correct the distorted lines.
• Figure 5: Example of our synthetic face pairs for training FaceCNet.