Joint Identity Verification and Pose Alignment for Partial Fingerprints

TL;DR

This work tackles partial fingerprint verification on compact sensors by proposing JIPNet, a CNN-Transformer hybrid that jointly performs identity verification and relative pose estimation, leveraging a cross-attentional interaction between paired fingerprints. A fingerprint enhancement pre-training task improves feature robustness for diverse textures and modalities. Extensive experiments across multiple public and in-house datasets show state-of-the-art performance and favorable efficiency, demonstrating the value of integrating pose information into the verification decision. The approach is particularly impactful for mobile biometrics, where partial data and posture variations are common, and it opens avenues for more robust one-to-one and one-to-many matching scenarios.

Abstract

Currently, portable electronic devices are becoming more and more popular. For lightweight considerations, their fingerprint recognition modules usually use limited-size sensors. However, partial fingerprints have few matchable features, especially when there are differences in finger pressing posture or image quality, which makes partial fingerprint verification challenging. Most existing methods regard fingerprint position rectification and identity verification as independent tasks, ignoring the coupling relationship between them -- relative pose estimation typically relies on paired features as anchors, and authentication accuracy tends to improve with more precise pose alignment. In this paper, we propose a novel framework for joint identity verification and pose alignment of partial fingerprint pairs, aiming to leverage their inherent correlation to improve each other. To achieve this, we present a multi-task CNN (Convolutional Neural Network)-Transformer hybrid network, and design a pre-training task to enhance the feature extraction capability. Experiments on multiple public datasets (NIST SD14, FVC2002 DB1A & DB3A, FVC2004 DB1A & DB2A, FVC2006 DB1A) and an in-house dataset show that our method achieves state-of-the-art performance in both partial fingerprint verification and relative pose estimation, while being more efficient than previous methods.

Figures (13)

• Figure 1: Partial fingerprint matching faces three challenges: sparse minutiae distribution, similar local texture and significant modal difference. This figure shows corresponding representative examples: (a) Three weakly-textured fingerprints that lack appropriate landmark feature points. (b) Local ridge patterns of fingerprints from different fingers have high similarity. (c) Fingerprint features in different skin conditions (in this case from left to right are dry, normal and wet respectively) may be missed (green) or incorrectly extracted (red). Visualized feature points are minutiae extracted by VeriFinger VeriFinger.
• Figure 2: Frameworks of different fingerprint matching algorithms. The process passed by parallel arrows indicates that the corresponding modules share weights and their functions can be executed independently. Feature Extraction in red and gold extract independent and interrelated features from paired data respectively. Pose Predictor in (a)(c)(d) and (b) estimate relative and absolute pose respectively.
• Figure 3: An overview of JIPNet. Paired fingerprint patches with the same shape are input, specifically $160 \times 160$, $120 \times 120$, or $96 \times 96$ in this paper. The outputs 'p' and 'H' of respective task heads correspond to the classification probability (whether the input fingerprints come from the same finger) and rigid transformation parameters (relative translation and rotation) respectively. Detailed structure is shown in Table \ref{['tab:network']}. Bars are presented on the left to indicate each phase, where the color definition refers to Fig. \ref{['fig:intro_method']}. The process passed by parallel arrows indicates that the corresponding modules share weights and their functions can be executed in parallel. * represents the number of channels are doubled after the corresponding convolution. Dotted parts are only used to assist training and could be pruned in practical verification tasks.
• Figure 4: Illustration of fingerprint enhancement task for pre-training. The network architecture is shown on the left, where solid and dotted boxes indicate the pre-trained parameters will (or not) be loaded into corresponding modules in JIPNet. Detailed structure is shown in Table \ref{['tab:network_pretrain']}. The right subfigure gives representative examples of inputs (original & augmented image) and targets (enhanced image) generation.
• Figure 5: Comparison of representation enhancement methods on four low-quality fingerprints. Each column from left to right is original image and corresponding enhancement results of CLAHE zuiderveld1994contrast, FingerNet tang2017fingernet, VeriFinger VeriFinger and our proposed method.