Gradient-Guided Exploration of Generative Model's Latent Space for Controlled Iris Image Augmentations

TL;DR

The paper addresses the need for diverse iris image augmentations with controllable attributes while preserving identity. It proposes gradient-guided traversals of a pre-trained iris image decoder's latent space, guided by differentiable attribute losses and an identity-preservation term, to move toward target iris features. The framework supports GAN inversion to manipulate real or synthetic iris images and presents a concrete set of differentiable losses (e.g., mask, sharpness, eyelid opening, pupil/iris sizes, PIR) integrated into a composite objective. Results demonstrate effective attribute manipulation (pupil/iris size, sharpness, PIR) with improved identity retention, and experiments show robustness across Z and W latent spaces, making the approach a flexible augmentation tool for iris recognition and presentation attack detection pipelines.

Abstract

Developing reliable iris recognition and presentation attack detection methods requires diverse datasets that capture realistic variations in iris features and a wide spectrum of anomalies. Because of the rich texture of iris images, which spans a wide range of spatial frequencies, synthesizing same-identity iris images while controlling specific attributes remains challenging. In this work, we introduce a new iris image augmentation strategy by traversing a generative model's latent space toward latent codes that represent same-identity samples but with some desired iris image properties manipulated. The latent space traversal is guided by a gradient of specific geometrical, textural, or quality-related iris image features (e.g., sharpness, pupil size, iris size, or pupil-to-iris ratio) and preserves the identity represented by the image being manipulated. The proposed approach can be easily extended to manipulate any attribute for which a differentiable loss term can be formulated. Additionally, our approach can use either randomly generated images using either a pre-train GAN model or real-world iris images. We can utilize GAN inversion to project any given iris image into the latent space and obtain its corresponding latent code.

Figures (5)

• Figure 1: We manipulate selected geometrical or textural iris image attribute by traversing the latent space of a generative decoder (trained to synthesize ISO/IEC 19794-6-compliant iris images). This traversal is guided by the gradient of a multi-term loss function (including a identity preservation component) with respect to the decoder's latent space.
• Figure 2: Comparison score distributions between initial iris image and the image with selected attributes manipulated, obtained with TripletNN matcher ND_OpenSourceIrisRecognition_GitHub in two scenarios: with and without inclusion of identity loss $\mathcal{L}_{{attr}:{id}}$. Plots were obtained for 400 comparison scores in each scenario: 10 random latent codes $\times$ 4 manipulated attributes (pupil size, iris size, sharpness and pupil-to-iris ratio) $\times$ 2 (decreasing and increasing the attribute's value) $\times$ 5 attribute's target values.
• Figure 3: Comparison between synthetic iris samples generated with and without use of identity loss component: (a) the initial iris sample., (b) the manipulated sample generated using identity loss, and (c) the manipulated iris sample generated without using identity loss.
• Figure 4: Illustration of the iris image attribute manipulation process through gradient-guided traversal of the $\mathcal{Z}$ latent space of the StyleGAN model trained for iris image synthesis, highlighting the effect of incorporating an identity loss term. The first row, bordered in blue, illustrates the resulting images when a decrease in the relevant attribute was requested. The second row, bordered in red, illustrates images with these attributes increased. The numerical values under each image indicate the comparison score obtained with the TripletNN matcher, and the corresponding attribute value obtained after latent space traversal.
• Figure 5: Same as in Figure \ref{['fig:iris_changes']}, except that the $\mathcal{W}$ latent space of StyleGAN2-ADA was used.