AS-FIBA: Adaptive Selective Frequency-Injection for Backdoor Attack on Deep Face Restoration

TL;DR

AS-FIBA tackles backdoor vulnerabilities in deep face restoration by introducing a degradation objective and an input-specific, frequency-domain trigger generator. The core method, SF-I-Net, performs adaptive frequency injection by dynamically decomposing features into frequency subbands and fusing trigger information via an SE-based mechanism, while a residual decoder hides the trigger under a perturbation constraint $||I - I_p||_\ell < \epsilon$. A pseudo-trigger training regime further enhances robustness against targeted backdoor patterns. Extensive experiments across multiple FR models (e.g., HiFaceGAN, GFP-GAN, VQFR, GPEN, CodeFormer, RestoreFormer) demonstrate high attack efficacy with minimal perceptual degradation, underscoring the vulnerability of FR systems to frequency-domain backdoors and providing a framework for evaluating defenses.

Abstract

Deep learning-based face restoration models, increasingly prevalent in smart devices, have become targets for sophisticated backdoor attacks. These attacks, through subtle trigger injection into input face images, can lead to unexpected restoration outcomes. Unlike conventional methods focused on classification tasks, our approach introduces a unique degradation objective tailored for attacking restoration models. Moreover, we propose the Adaptive Selective Frequency Injection Backdoor Attack (AS-FIBA) framework, employing a neural network for input-specific trigger generation in the frequency domain, seamlessly blending triggers with benign images. This results in imperceptible yet effective attacks, guiding restoration predictions towards subtly degraded outputs rather than conspicuous targets. Extensive experiments demonstrate the efficacy of the degradation objective on state-of-the-art face restoration models. Additionally, it is notable that AS-FIBA can insert effective backdoors that are more imperceptible than existing backdoor attack methods, including WaNet, ISSBA, and FIBA.

Figures (7)

• Figure 1: Illustration of imperceptible backdoor attack on the face restoration model. When inputting the benign image, a high-quality image can be restored. Whereas a degraded image is generated by inputting the poisoned image.
• Figure 2: The architecture of the proposed selective frequency-injection network (SF-I-Net).
• Figure 3: Visualization of HiFaceGAN output under different backdoor attack methods for benign and poisoned inputs. The rows represent different levels of neuron pruning: 10%, 50%, and 90%.
• Figure 4: Visualization for results of the victim HiFaceGAN under different backdoor attacks. The first column displays benign inputs and corresponding outputs on the clean model, while the subsequent four columns show poisoned inputs and attacked outputs of different backdoors. Best view by zooming in.
• Figure 5: Benign accuracy (BA) and attack success rate (ASR) of different attack methods against pruning-based defense.