Deep Generative Attacks and Countermeasures for Data-Driven Offline Signature Verification

TL;DR

The paper addresses the vulnerability of data-driven offline signature verification to deep generative attacks using $VAE$ and $CGAN$, quantifying synthetic signature quality with $SSIM$ and evaluating across multiple architectures and three datasets. It reveals that while baseline systems achieve low $FAR$ on genuine vs forgery tests, synthetic signatures can dramatically increase $FAR$, especially for $CGAN$-generated forgeries, with $FAR$ reaching up to $61.64\%$. A key contribution is the SSIM-controlled retraining approach, which, particularly with $VAE$-SSI data, reduces $FAR$ to 0–0.99\% across attack scenarios, demonstrating a practical defense against evolving generative threats. The findings underscore the importance of integrating synthetic-forgery exposure into training and point to future work on additional image quality metrics and extensions to online and writer-independent verification to bolster real-world security.

Abstract

This study investigates the vulnerabilities of data-driven offline signature verification (DASV) systems to generative attacks and proposes robust countermeasures. Specifically, we explore the efficacy of Variational Autoencoders (VAEs) and Conditional Generative Adversarial Networks (CGANs) in creating deceptive signatures that challenge DASV systems. Using the Structural Similarity Index (SSIM) to evaluate the quality of forged signatures, we assess their impact on DASV systems built with Xception, ResNet152V2, and DenseNet201 architectures. Initial results showed False Accept Rates (FARs) ranging from 0% to 5.47% across all models and datasets. However, exposure to synthetic signatures significantly increased FARs, with rates ranging from 19.12% to 61.64%. The proposed countermeasure, i.e., retraining the models with real + synthetic datasets, was very effective, reducing FARs between 0% and 0.99%. These findings emphasize the necessity of investigating vulnerabilities in security systems like DASV and reinforce the role of generative methods in enhancing the security of data-driven systems.

Figures (6)

• Figure 1: Illustrating the process from data generation to model evaluation. Data sources include human-generated signatures, random noise, and deep generative models (DGMs), creating training and testing datasets. During training, baseline models are trained. In testing, models are first evaluated on accepting genuine and rejecting forged signatures, then tested for susceptibility to generated signatures. Models are then retrained with generated samples and re-evaluated for both criteria. Performance is measured using false reject rates (FRR) and false accept rates (FAR). Solid arrows represent baseline training, while dashed arrows indicate retraining.
• Figure 2: The Pearson correlation coefficient ($r$) of $-0.8640$ and $R^2$ of $0.7465$ demonstrate a strong negative correlation between SSIM and FAR, with a p-value of 6.39e-09 confirming statistical significance. This means lower-quality synthetic forgeries (lower SSIM) are more likely to be accepted as genuine by the verification system because the generated samples are farther from the forgeries and, in turn, closer to genuine signatures in a two-class setup. This observation formed the basis for a novel countermeasure, i.e., retraining models with SSIM-controlled datasets, to enhance the robustness of DASV systems against synthetic forgeries.
• Figure 3: The heatmap of SSIMs achieved by CGAN and VAE-based signature generators across different epochs. On the y-axis are 275 signature samples generated by each generator, with five samples per user. VAE could generate signatures with a wide range of SSIMs, i.e., signatures close to and far from the reference signatures (skilled forgeries) across varying epochs. In contrast, CGAN could not match VAE's capability to generate a variety of signatures with differing SSIMs. In other words, VAE outperformed CGAN in controlled signature generation, thus chosen for further experiments.
• Figure 4: Radar chart illustrating the performance of three baseline models—DenseNet201, ResNet152V2, and Xception—on the CEDAR, BHSig260-Bengali (Bengali), and BHSig260-Hindi (Hindi) datasets. The axes represent each dataset's False Accept Rates (FAR) and False Reject Rates (FRR). All models achieved error rates below 6% for both FAR and FRR. DenseNet201 performed best with FARs and FRRs lower than 2% across datasets, followed by ResNet152V2 and Xception.
• Figure 5: Attack performance on the baseline ASVs. Synthetic signatures were generated using skilled forgeries as a reference, and each DGM ran for $800$ epochs. The higher the SSIM, the more similar the generated signature is to the forgeries, resulting in a lower FAR. Conversely, the lower the SSIM, the farther the generated signature is from the reference forgeries, resulting in a higher FAR. Random signature attacks resulted in high vulnerability, with FARs over 30% on average. VAE attacks produced FARs closest to original forgeries, except for the CEDAR models, which had higher FARs around 20-25%. CGAN attacks caused the highest FARs, exceeding 29% on average across all models and datasets.