Data Distribution Shifts in (Industrial) Federated Learning as a Privacy Issue

TL;DR

In an empirical study on benchmark datasets, an honest-but-curious attacker is shown to be capable of detecting subtle distributional shifts on other clients, in some cases long before they become obvious in evaluation.

Abstract

We consider industrial federated learning, a collaboration between a small number of powerful, potentially competing industrial players, mediated by a third party aspiring to improve the service it provides to its customers. We argue that this configuration harbours covert privacy risks that do not arise in e.g. cross-device settings. Companies are very protective of their intellectual property and production processes. Information about changes to their production and the timing of which is to be kept private. We study a scenario in which one of the collaborators infers changes to their competitors' production by detecting potentially subtle temporal data distribution shifts. In this framing, a data distribution shift is always problematic, even if it has no negative effect on training convergence. Thus, our goal is to find means that allow the detection of distributional shifts better than customary evaluation metrics. Based on the assumption that even minor shifts translate into the collaboratively learned machine learning model, the attacker tracks the shared models' internal state with a selection of metrics from literature in order to pick up on relevant changes. In an empirical study on benchmark datasets, we show an honest-but-curious attacker to be capable of detecting subtle distributional shifts on other clients, in some cases long before they become obvious in evaluation.

Figures (10)

• Figure 1: High-level overview over the proposed attack on industrial FL. An attacker infers a data distribution shift on the other client occurring at some point during the FL. The shift is subtle enough so as to not affect conventional evaluation metrics, but can still be detected via more specialized metrics.
• Figure 2: Left: A company sells devices that assist the manufacturing of industrial components and allow the assessment of their quality. The company adds AI-based predictive services which allows the manufacturers to predict defects ahead of time by using accrued process- and quality data for training a model. Since the company's customers have similar tasks they could offer to join a collaborative scheme across manufacturers to enhance the predictive performance. Right: A label distribution shift can occur if a manufacturer produces a new component which affects the distribution of identified defects. The detection of these changes by other clients could pose a privacy risk by disclosing operational adjustments.
• Figure 3: Available information for an attacker in FL. The attacker cannot access the data of another client directly, but the global model's internal state is accessible, including the weights, the internal representations and the gradients. Ideally, these encode enough evidence of a DDS for the attacker to extract.
• Figure 4: Timeline of attacker information acquisition. In the second round the attacker can extract the target weights and obtain the representations, and approximate the gradients in the third. Their trend can then be determined one round later respectively, by comparing the values of subsequent rounds. Therefore, a DDS can be detected by means of weight- and representation trend from round two forward and via the gradients starting from round three. Since the global weights of any specific round always encode the contributions of the clients from the round before, the detection of a DDS always has a one round delay.
• Figure 5: Results of the centralized setting. The plots show a study of the SoL on a NN trained on MNIST, experiencing a label distribution shift at round 11. All results are reported as mean and standard deviation across three runs.