Finger Pose Estimation for Under-screen Fingerprint Sensor

TL;DR

This paper tackles the challenge of estimating finger pose for partial fingerprints captured by under-screen sensors. It introduces DRACO, a dual-modal framework that simultaneously leverages ridge patches and capacitive images, fusing their complementary information through a Mixture of Experts and a knowledge-transfer mechanism from plain to partial fingerprints. Pose is represented as decoupled probability distributions over $x$, $y$, $ ext{cos} heta$, and $ ext{sin} heta$, enabling robust, differentiable regression via expectation over class embeddings. The approach achieves state-of-the-art performance on multiple public and private datasets, improving both pose estimation accuracy and downstream fingerprint recognition tasks, and demonstrates practical feasibility with competitive efficiency. The work provides a principled direction for integrating multimodal, partial-sensor data into robust biometric pipelines.

Abstract

Two-dimensional pose estimation plays a crucial role in fingerprint recognition by facilitating global alignment and reduce pose-induced variations. However, existing methods are still unsatisfactory when handling with large angle or small area inputs. These limitations are particularly pronounced on fingerprints captured by under-screen fingerprint sensors in smartphones. In this paper, we present a novel dual-modal input based network for under-screen fingerprint pose estimation. Our approach effectively integrates two distinct yet complementary modalities: texture details extracted from ridge patches through the under-screen fingerprint sensor, and rough contours derived from capacitive images obtained via the touch screen. This collaborative integration endows our network with more comprehensive and discriminative information, substantially improving the accuracy and stability of pose estimation. A decoupled probability distribution prediction task is designed, instead of the traditional supervised forms of numerical regression or heatmap voting, to facilitate the training process. Additionally, we incorporate a Mixture of Experts (MoE) based feature fusion mechanism and a relationship driven cross-domain knowledge transfer strategy to further strengthen feature extraction and fusion capabilities. Extensive experiments are conducted on several public datasets and two private datasets. The results indicate that our method is significantly superior to previous state-of-the-art (SOTA) methods and remarkably boosts the recognition ability of fingerprint recognition algorithms. Our code is available at https://github.com/XiongjunGuan/DRACO.

Figures (11)

• Figure 1: Examples of fingerprint pose estimation under different input modalities. Among them, full fingerprint (plain fingerprint) is collected by a conventional optical fingerprint scanner, while the ridge patch and capacitive image are simultaneously collected from the under screen fingerprint sensor and touch screen of a smartphone (referred to as partial fingerprint collectively) . All modalities, captured from the same finger with similar touch gestures, are marked in gray, purple, and green. For clarity, the dashed lines in the full fingerprint indicate the equivalent collection areas for the partial fingerprints. Subfigures in the last row represent the estimated result (blue) and ground truth (red) using corresponding modals. It can be observed that the performance of previous fingerprint based SOTA solution duan2023estimating declines significantly as the available area diminishes, while our dual-modal method achieves more accurate prediction.
• Figure .1: The VeriFinger verifinger based fingerprint indexing performance with corresponding pose constraint on full fingerprints from DPF under different rotation ranges: (a) $[45^\circ,45^\circ]$, (b) $[90^\circ,90^\circ]$, (c) $[135^\circ,135^\circ]$, (d) $[180^\circ,180^\circ]$. Different input modalities are distinguished by the shape of markers.
• Figure 2: An overview of our partial fingerprint pose estimation network DRACO. The ridge patch and capacitive image collected simultaneously from the touch device equipped with an under screen fingerprint sensor are input. The prediction results are represented by the horizontal and vertical coordinates of the center, as well as the sine and cosine values of the direction.
• Figure 3: The main process of knowledge transfer during network training. The blue and yellow modules corresponds to modules of the same color in Fig. \ref{['fig:network']}. As shown in the bottom right corner, the features of dual modal inputs are progressively aligned with the depiction in plain fingerprints through contrastive learning techniques. The gray dashed line represents the process of data synthesis.
• Figure 4: Image examples from different datasets: (a) FVC2002 DB1_A fvc2002, (b) FVC2004 DB1_A fvc2004, (c) DPF duan2023estimating, rolled fingerprint, (d) DPF duan2023estimating, plain fingerprints, (e) DPF duan2023estimating, simulated partial fingerprints, (f) PCF, rolled fingerprints, (g) PCF, partial fingerprints. The 'partial fingerprint' is a general term used to represent two modalities: ridge patch and capacitive image.