Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SEBA: Strong Evaluation of Biometric Anonymizations

Julian Todt, Simon Hanisch, Thorsten Strufe

TL;DR

SEBA addresses the need for strong evaluation of biometric anonymizations by integrating recent advances in attacker modeling, identity-set reduction, and de-anonymization into a single software framework. The framework automates data management, anonymization, de-anonymization, recognition, and metric visualization, enabling rigorous privacy-utility analysis across multiple biometric traits. The authors also introduce new metrics and propose plotting privacy versus utility with area under the curve to enable fair comparisons. A prototypical CelebA-based experiment demonstrates SEBA's applicability, showing how state-of-the-art anonymization like DeepPrivacy can preserve utility while other methods trade privacy for minimal gains, underscoring the importance of worst-case evaluation. SEBA's open-source design aims to standardize and streamline robust evaluation for designers of biometric anonymizations.

Abstract

Biometric data is pervasively captured and analyzed. Using modern machine learning approaches, identity and attribute inferences attacks have proven high accuracy. Anonymizations aim to mitigate such disclosures by modifying data in a way that prevents identification. However, the effectiveness of some anonymizations is unclear. Therefore, improvements of the corresponding evaluation methodology have been proposed recently. In this paper, we introduce SEBA, a framework for strong evaluation of biometric anonymizations. It combines and implements the state-of-the-art methodology in an easy-to-use and easy-to-expand software framework. This allows anonymization designers to easily test their techniques using a strong evaluation methodology. As part of this discourse, we introduce and discuss new metrics that allow for a more straightforward evaluation of the privacy-utility trade-off that is inherent to anonymization attempts. Finally, we report on a prototypical experiment to demonstrate SEBA's applicability.

SEBA: Strong Evaluation of Biometric Anonymizations

TL;DR

SEBA addresses the need for strong evaluation of biometric anonymizations by integrating recent advances in attacker modeling, identity-set reduction, and de-anonymization into a single software framework. The framework automates data management, anonymization, de-anonymization, recognition, and metric visualization, enabling rigorous privacy-utility analysis across multiple biometric traits. The authors also introduce new metrics and propose plotting privacy versus utility with area under the curve to enable fair comparisons. A prototypical CelebA-based experiment demonstrates SEBA's applicability, showing how state-of-the-art anonymization like DeepPrivacy can preserve utility while other methods trade privacy for minimal gains, underscoring the importance of worst-case evaluation. SEBA's open-source design aims to standardize and streamline robust evaluation for designers of biometric anonymizations.

Abstract

Biometric data is pervasively captured and analyzed. Using modern machine learning approaches, identity and attribute inferences attacks have proven high accuracy. Anonymizations aim to mitigate such disclosures by modifying data in a way that prevents identification. However, the effectiveness of some anonymizations is unclear. Therefore, improvements of the corresponding evaluation methodology have been proposed recently. In this paper, we introduce SEBA, a framework for strong evaluation of biometric anonymizations. It combines and implements the state-of-the-art methodology in an easy-to-use and easy-to-expand software framework. This allows anonymization designers to easily test their techniques using a strong evaluation methodology. As part of this discourse, we introduce and discuss new metrics that allow for a more straightforward evaluation of the privacy-utility trade-off that is inherent to anonymization attempts. Finally, we report on a prototypical experiment to demonstrate SEBA's applicability.
Paper Structure (27 sections, 4 figures)

This paper contains 27 sections, 4 figures.

Table of Contents

  1. Introduction
  2. Background
  3. Biometrics, Identification & Anonymization
  4. Recent Evaluation Methodology Advances
  5. Related Work
  6. Motivation and Goals
  7. Adversary Model
  8. Evaluation Methodology
  9. Rigorous Privacy Evaluation
  10. Measuring Utility
  11. Fair Comparison & Metrics
  12. Design
  13. Implementation
  14. Sample Experiment
  15. Conclusion
  16. ...and 12 more sections

Figures (4)

  • Figure 1: Design of our evaluation framework (simplified).
  • Figure 2: Results for privacy and utility for DeepPrivacy, Blurring and Eye Masking on 100 identities of CelebA without de-anonymization. Clear-level utility and change-level privacy are indicated with dotted lines.
  • Figure 3: Results for privacy and utility for DeepPrivacy, Blurring and Eye Masking on 100 identities of CelebA with de-anonymization. Clear-level utility and change-level privacy are indicated with dotted lines.
  • Figure 4: Tested anonymizations in our sample experiment