Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Watchlist Challenge: 3rd Open-set Face Detection and Identification

Furkan Kasım, Terrance E. Boult, Rensso Mora, Bernardo Biesseck, Rafael Ribeiro, Jan Schlueter, Tomáš Repák, Rafael Henrique Vareto, David Menotti, William Robson Schwartz, Manuel Günther

TL;DR

This paper presents a comprehensive evaluation of participating algorithms, using the enhanced UnConstrained College Students (UCCS) dataset with new evaluation protocols, demonstrating that while detection capabilities are generally robust, closed-set identification performance varies significantly.

Abstract

In the current landscape of biometrics and surveillance, the ability to accurately recognize faces in uncontrolled settings is paramount. The Watchlist Challenge addresses this critical need by focusing on face detection and open-set identification in real-world surveillance scenarios. This paper presents a comprehensive evaluation of participating algorithms, using the enhanced UnConstrained College Students (UCCS) dataset with new evaluation protocols. In total, four participants submitted four face detection and nine open-set face recognition systems. The evaluation demonstrates that while detection capabilities are generally robust, closed-set identification performance varies significantly, with models pre-trained on large-scale datasets showing superior performance. However, open-set scenarios require further improvement, especially at higher true positive identification rates, i.e., lower thresholds.

Watchlist Challenge: 3rd Open-set Face Detection and Identification

TL;DR

This paper presents a comprehensive evaluation of participating algorithms, using the enhanced UnConstrained College Students (UCCS) dataset with new evaluation protocols, demonstrating that while detection capabilities are generally robust, closed-set identification performance varies significantly.

Abstract

In the current landscape of biometrics and surveillance, the ability to accurately recognize faces in uncontrolled settings is paramount. The Watchlist Challenge addresses this critical need by focusing on face detection and open-set identification in real-world surveillance scenarios. This paper presents a comprehensive evaluation of participating algorithms, using the enhanced UnConstrained College Students (UCCS) dataset with new evaluation protocols. In total, four participants submitted four face detection and nine open-set face recognition systems. The evaluation demonstrates that while detection capabilities are generally robust, closed-set identification performance varies significantly, with models pre-trained on large-scale datasets showing superior performance. However, open-set scenarios require further improvement, especially at higher true positive identification rates, i.e., lower thresholds.
Paper Structure (15 sections, 6 equations, 4 figures, 3 tables)

This paper contains 15 sections, 6 equations, 4 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Dataset and Protocol
  3. Defining new Evaluation Protocols
  4. Challenge Participants
  5. Face Detection
  6. Face Identification
  7. Joint Face Detection and Identification
  8. Evaluation
  9. Face Detection
  10. Face Identification
  11. Discussion
  12. Closed-Set Performance
  13. Analysis of Threshold
  14. Analysis of the Unknown
  15. Conclusion

Figures (4)

  • Figure 1: Example Images and Watchlist. \ref{['fig:UCCS:images']} shows two images with their annotations from the new version of UCCS dataset, including occlusions, different angles, and instances of significant blur. Faces marked with the same color indicate the same identity, whereas white boxes denote unknown subjects. \ref{['fig:UCCS:watchlist']} displays cropped faces in the watchlist, including 5 facial landmarks.
  • Figure 2: Face Detection and Recognition Evaluation. A Free-response Receiver Operating Characteristic (FROC) curve is shown for the \ref{['fig:FROC:validation']} validation and \ref{['fig:FROC:test']} test set. The horizontal axis includes the number of false positive detections normalized by the number of images, while the vertical axis outlines the relative number of true positive detections of faces. Open-set ROC curve at rank 1 is shown for \ref{['fig:OROC:validation']} validation and \ref{['fig:OROC:test']} test set. The horizontal axis includes the number of false positive identifications normalized by the number of images, while the vertical axis outlines the relative number of correctly identified faces.
  • Figure 3: Threshold Selection. We depict the effect of selecting the thresholds on the validation and test sets. In \ref{['fig:thresholds:detection']}, we show differences in detection scores, while \ref{['fig:thresholds:recognition']} highlights differences in identification performances.
  • Figure 4: Rejection Rates by Type. Unknown samples are split into \ref{['fig:CCR:unknown']} unknown subjects and \ref{['fig:CCR:background']} false positive detections, illustrating the rate of correctly rejected samples (i.e., samples not identified as any known subject) across varying thresholds that are based on the number of correctly identified known subjects per image. X-axes are plotted in logarithmic scale.