Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Which PPML Would a User Choose? A Structured Decision Support Framework for Developers to Rank PPML Techniques Based on User Acceptance Criteria

Sascha Löbner, Sebastian Pape, Vanessa Bracamonte, Kittiphop Phalakarn

TL;DR

This study introduces a decision support framework with the aim of supporting the selection of PPML technologies based on user preferences and achieves a technology ranking based on the User Acceptance Criteria while providing technology insights for the developers.

Abstract

Using Privacy-Enhancing Technologies (PETs) for machine learning often influences the characteristics of a machine learning approach, e.g., the needed computational power, timing of the answers or how the data can be utilized. When designing a new service, the developer faces the problem that some decisions require a trade-off. For example, the use of a PET may cause a delay in the responses or adding noise to the data to improve the users' privacy might have a negative impact on the accuracy of the machine learning approach. As of now, there is no structured way how the users' perception of a machine learning based service can contribute to the selection of Privacy Preserving Machine Learning (PPML) methods. This is especially a challenge since one cannot assume that users have a deep technical understanding of these technologies. Therefore, they can only be asked about certain attributes that they can perceive when using the service and not directly which PPML they prefer. This study introduces a decision support framework with the aim of supporting the selection of PPML technologies based on user preferences. Based on prior work analysing User Acceptance Criteria (UAC), we translate these criteria into differentiating characteristics for various PPML techniques. As a final result, we achieve a technology ranking based on the User Acceptance Criteria while providing technology insights for the developers. We demonstrate its application using the use case of classifying privacy-relevant information. Our contribution consists of the decision support framework which consists of a process to connect PPML technologies with UAC, a process for evaluating the characteristics that separate PPML techniques, and a ranking method to evaluate the best PPML technique for the use case.

Which PPML Would a User Choose? A Structured Decision Support Framework for Developers to Rank PPML Techniques Based on User Acceptance Criteria

TL;DR

This study introduces a decision support framework with the aim of supporting the selection of PPML technologies based on user preferences and achieves a technology ranking based on the User Acceptance Criteria while providing technology insights for the developers.

Abstract

Using Privacy-Enhancing Technologies (PETs) for machine learning often influences the characteristics of a machine learning approach, e.g., the needed computational power, timing of the answers or how the data can be utilized. When designing a new service, the developer faces the problem that some decisions require a trade-off. For example, the use of a PET may cause a delay in the responses or adding noise to the data to improve the users' privacy might have a negative impact on the accuracy of the machine learning approach. As of now, there is no structured way how the users' perception of a machine learning based service can contribute to the selection of Privacy Preserving Machine Learning (PPML) methods. This is especially a challenge since one cannot assume that users have a deep technical understanding of these technologies. Therefore, they can only be asked about certain attributes that they can perceive when using the service and not directly which PPML they prefer. This study introduces a decision support framework with the aim of supporting the selection of PPML technologies based on user preferences. Based on prior work analysing User Acceptance Criteria (UAC), we translate these criteria into differentiating characteristics for various PPML techniques. As a final result, we achieve a technology ranking based on the User Acceptance Criteria while providing technology insights for the developers. We demonstrate its application using the use case of classifying privacy-relevant information. Our contribution consists of the decision support framework which consists of a process to connect PPML technologies with UAC, a process for evaluating the characteristics that separate PPML techniques, and a ranking method to evaluate the best PPML technique for the use case.

Paper Structure

This paper contains 37 sections, 6 equations, 4 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Related Literature
  3. Methodology
  4. Entities Involved
  5. Research Objectives
  6. Threat Model
  7. Framework Description
  8. Expert Input and Framework Setup
  9. User Input
  10. Developer Input
  11. Assessing eligible PPML techniques
  12. Setting PPML categories for application
  13. Setting weights for categories
  14. Evaluating PPML categories
  15. Framework Mapping
  16. ...and 22 more sections

Figures (4)

  • Figure 1: Flowchart illustrating the procedural steps for implementing the framework. The sections of the description (section \ref{['sec:framew_descr']}) correspond to the framework application (section \ref{['sec:application']}).
  • Figure 2: Example of a privacy tool for PSI detection in social media texts. bracamonte2022all
  • Figure 3: Importance scores $u_{j}$ (x-axis) of UAC (y-axis)
  • Figure 4: Importance scores $\tilde{c}_i$ (x-axis) of PPML Characteristics (y-axis) for user/data entity