PrivatEyes: Appearance-based Gaze Estimation Using Federated Secure Multi-Party Computation

TL;DR

Evaluations on the MPIIGaze, MPIIFaceGaze, GazeCapture, and NVGaze datasets show that the improved privacy does not lead to a lower gaze estimation accuracy or substantially higher computational costs - both of which are on par with its non-secure counterparts.

Abstract

Latest gaze estimation methods require large-scale training data but their collection and exchange pose significant privacy risks. We propose PrivatEyes - the first privacy-enhancing training approach for appearance-based gaze estimation based on federated learning (FL) and secure multi-party computation (MPC). PrivatEyes enables training gaze estimators on multiple local datasets across different users and server-based secure aggregation of the individual estimators' updates. PrivatEyes guarantees that individual gaze data remains private even if a majority of the aggregating servers is malicious. We also introduce a new data leakage attack DualView that shows that PrivatEyes limits the leakage of private training data more effectively than previous approaches. Evaluations on the MPIIGaze, MPIIFaceGaze, GazeCapture, and NVGaze datasets further show that the improved privacy does not lead to a lower gaze estimation accuracy or substantially higher computational costs - both of which are on par with its non-secure counterparts.

Figures (9)

• Figure 1: PrivatEyes combines federated learning (FL) and secure multi-party computation (MPC) for privacy-enhancing training of appearance-based gaze estimation methods (see \ref{['sec:background', 'sub: privatEyesTrain']}). Clients $C_j$ locally train the gaze estimation model on their private data. Each client $C_j$ splits her individually trained model parameters (IU) $a_j$ into $n$ secret shares, i.e. in this example $n=3$. Each server $S_i,1\leq i,\leq n$ receives its respective share of each $a_j$, e.g. $S_1$ receives the share $5$ of $a_1=3$, $3$ of $a_2=10$ and $16$ of $a_3=8$. Then each server aggregates the different shares, e.g. $S_1$ computes $5+3+16 \operatorname{mod} 23=1$, and sends the result (i.e. output model OM) to all clients. The clients compute the average $(1+9+11)/3=7$, but nothing more.
• Figure 2: DualView demonstrates the amount of gaze-specific information leakage by reconstructing the user’s appearance (View1: how the user looks like) as well as inferring the corresponding gaze distribution (View2: where the user is looking) of the private dataset used to train a target gaze estimation model.
• Figure 3: Sample reconstruction results. (A) Reconstructed face/eye images of the same participant for the different baselines (Data centre is the ground truth due to the direct access to the raw data). (B) Effect of the number of training samples per round. (C) Using previous reconstructions as auxiliary knowledge against adaptive FL training affects the reconstruction. (D) Effect of the attack loss function, e.g. transferring the gaze of a victim to a random face (1st row) and transferring the appearance of a victim to a random gaze (2nd row) in adaptive FL.
• Figure 4: For similar ground truth (left) and reconstructed images (right) from the 3 different datasets, participants were asked the following questions: A.1. Do both images represent the same person? (Yes/No) A.2. On a scale from 1 to 10, how would you rate the overall similarity of both images? (Very similar to very dissimilar)
• Figure 5: For similar reconstructed images (left), participants were asked to map the full-face/eye image to the corresponding ground-truth clients.