Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LocalScore: Local Density-Aware Similarity Scoring for Biometrics

Yiyang Su, Minchul Kim, Jie Zhu, Christopher Perry, Feng Liu, Anil Jain, Xiaoming Liu

TL;DR

This work tackles open-set biometrics by addressing the limitation of collapsing multi-sample gallery variability into a single prototype. It introduces LocalScore, a simple, architecture-agnostic scoring method that augments traditional per-subject scores with a k-th nearest neighbor-based local-density term, computed per probe and selectively fused to the top-scoring subject. The authors provide a theoretical framework with conditions under which LocalScore improves FNIR@FPIR and TAR@FAR, and validate the approach across face, gait, and person reID tasks using diverse models and datasets, including Monte Carlo simulations that support their theory. Practically, LocalScore delivers substantial gains with negligible overhead and supports optional gallery clustering to balance accuracy and efficiency, making it a ready-to-deploy enhancement for real-world open-set biometric systems.

Abstract

Open-set biometrics faces challenges with probe subjects who may not be enrolled in the gallery, as traditional biometric systems struggle to detect these non-mated probes. Despite the growing prevalence of multi-sample galleries in real-world deployments, most existing methods collapse intra-subject variability into a single global representation, leading to suboptimal decision boundaries and poor open-set robustness. To address this issue, we propose LocalScore, a simple yet effective scoring algorithm that explicitly incorporates the local density of the gallery feature distribution using the k-th nearest neighbors. LocalScore is architecture-agnostic, loss-independent, and incurs negligible computational overhead, making it a plug-and-play solution for existing biometric systems. Extensive experiments across multiple modalities demonstrate that LocalScore consistently achieves substantial gains in open-set retrieval (FNIR@FPIR reduced from 53% to 40%) and verification (TAR@FAR improved from 51% to 74%). We further provide theoretical analysis and empirical validation explaining when and why the method achieves the most significant gains based on dataset characteristics.

LocalScore: Local Density-Aware Similarity Scoring for Biometrics

TL;DR

This work tackles open-set biometrics by addressing the limitation of collapsing multi-sample gallery variability into a single prototype. It introduces LocalScore, a simple, architecture-agnostic scoring method that augments traditional per-subject scores with a k-th nearest neighbor-based local-density term, computed per probe and selectively fused to the top-scoring subject. The authors provide a theoretical framework with conditions under which LocalScore improves FNIR@FPIR and TAR@FAR, and validate the approach across face, gait, and person reID tasks using diverse models and datasets, including Monte Carlo simulations that support their theory. Practically, LocalScore delivers substantial gains with negligible overhead and supports optional gallery clustering to balance accuracy and efficiency, making it a ready-to-deploy enhancement for real-world open-set biometric systems.

Abstract

Open-set biometrics faces challenges with probe subjects who may not be enrolled in the gallery, as traditional biometric systems struggle to detect these non-mated probes. Despite the growing prevalence of multi-sample galleries in real-world deployments, most existing methods collapse intra-subject variability into a single global representation, leading to suboptimal decision boundaries and poor open-set robustness. To address this issue, we propose LocalScore, a simple yet effective scoring algorithm that explicitly incorporates the local density of the gallery feature distribution using the k-th nearest neighbors. LocalScore is architecture-agnostic, loss-independent, and incurs negligible computational overhead, making it a plug-and-play solution for existing biometric systems. Extensive experiments across multiple modalities demonstrate that LocalScore consistently achieves substantial gains in open-set retrieval (FNIR@FPIR reduced from 53% to 40%) and verification (TAR@FAR improved from 51% to 74%). We further provide theoretical analysis and empirical validation explaining when and why the method achieves the most significant gains based on dataset characteristics.
Paper Structure (45 sections, 3 theorems, 21 equations, 11 figures, 8 tables)

This paper contains 45 sections, 3 theorems, 21 equations, 11 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Biometrics.
  4. Open-Set Recognition.
  5. Re-ranking.
  6. Method
  7. Problem Setup
  8. LocalScore
  9. Gallery Clustering
  10. Theoretical Analyses
  11. Experiments
  12. Evaluation Protocols.
  13. Evaluation Metrics.
  14. Datasets.
  15. Ablation Studies
  16. ...and 30 more sections

Key Result

Theorem 1

Alg. alg:main_alg improves the expected open-set FNIR@FPIR at the expected threshold if where $\delta = -\ln (-\ln(1 - r_1 / N_2))$ and improves verification TAR@FAR at the expected threshold if where $\Phi^{-1}(\cdot)$ is the inverse of the cumulative distribution function of normal distribution, and $\alpha=\pi/8$ is a constant.

Figures (11)

  • Figure 1: Illustration of the effect of local density in face recognition. Each face represents a gallery feature, and distinct clusters correspond to different viewpoints. Conventional biometric systems only represent each subject by a single class center (shown as the stars). However, a probe sample may lie close to this center yet remain far from all actual clusters, leading to false acceptance in verification or open-set retrieval. In contrast, the local density of the gallery feature distribution captures the true intra-subject variability and correctly rejects such non-mated probes.
  • Figure 2: Algorithm 1. Python-style pseudo-code for LocalScore.
  • Figure 2: A toy example of LocalScore, where genuine scores are bolded and underscored. Different colors represent different gallery subjects. The $k$-NN scores are calculated from the per-media score matrix for each probe and added to the maximum per-subject score to generate the output score matrix. For instance, for the first probe, a $k$-NN score of $0.95$ is added to the highest per-subject score of $0.7$, while all other scores remain unchanged. In the original score matrix, only one genuine score exceeds the maximum-per-probe non-mated score of $0.8$. However, after applying LocalScore, all genuine scores surpass the updated maximum-per-probe non-mated score of $1.4$.
  • Figure 3: The effect of $k$ on the overall performance. The total performance is the sum of TPIR@FPIR (i.e., $1 -$FNIR@FPIR), TAR@FAR, and rank-1 accuracy. Dashed lines: without LocalScore.
  • Figure 4: The relationship between performance gains from LocalScore on various datasets and the theoretical difference between the right-hand and the left-hand side of \ref{['eq:open_set_theory']}. Red line: linear regression.
  • ...and 6 more figures

Theorems & Definitions (5)

  • Theorem 1
  • Theorem 2
  • proof
  • Theorem 1
  • proof