TOFU: Towards Obfuscated Federated Updates by Encoding Weight Updates into Gradients from Proxy Data

TL;DR

This work tackles the dual challenges of communication efficiency and gradient-based data leakage in Federated Learning by introducing TOFU, a method that encodes each client’s weight update into a small set of proxy data whose gradients reproduce the update. The synthetic proxy data resemble noise, reducing leakage risk, and the approach yields substantial per-round communication savings while maintaining near-baseline accuracy; any accuracy loss incurred by using proxy data can be recovered with a small number of encrypted full-gradient rounds. The method is formalized by defining a synthetic dataset that encodes updates via a gradient-alignment objective, and is deployed through a four-phase federated protocol that combines up- and down-communication via proxy data. Experiments on MNIST and CIFAR-10 show that TOFU achieves up to around 4×–6.6× communication efficiency with minimal accuracy degradation, and that full accuracy can be restored with only a handful of additional secure communication rounds, making the approach practical for larger, more complex models.

Abstract

Advances in Federated Learning and an abundance of user data have enabled rich collaborative learning between multiple clients, without sharing user data. This is done via a central server that aggregates learning in the form of weight updates. However, this comes at the cost of repeated expensive communication between the clients and the server, and concerns about compromised user privacy. The inversion of gradients into the data that generated them is termed data leakage. Encryption techniques can be used to counter this leakage, but at added expense. To address these challenges of communication efficiency and privacy, we propose TOFU, a novel algorithm which generates proxy data that encodes the weight updates for each client in its gradients. Instead of weight updates, this proxy data is now shared. Since input data is far lower in dimensional complexity than weights, this encoding allows us to send much lesser data per communication round. Additionally, the proxy data resembles noise, and even perfect reconstruction from data leakage attacks would invert the decoded gradients into unrecognizable noise, enhancing privacy. We show that TOFU enables learning with less than 1% and 7% accuracy drops on MNIST and on CIFAR-10 datasets, respectively. This drop can be recovered via a few rounds of expensive encrypted gradient exchange. This enables us to learn to near-full accuracy in a federated setup, while being 4x and 6.6x more communication efficient than the standard Federated Averaging algorithm on MNIST and CIFAR-10, respectively.

Figures (7)

• Figure 1: A pictorial representation of our encoding. The loss landscape shown in blue is taken from landscape, with the starting point marked with a red circle. Each client learns on some minibatches of real data, visualized on the top. The updates from these minibatches are shown with red arrows. The final weight update to be encoded and communicated, $U_{real}$, is calculated and shown in white. We construct a limited set of synthetic data that generates gradients $g_{syn}$ on the loss landscape, a weighted combination of which results in $U_{syn}$. The reconstruction algorithm optimizes these images and weights (denoted by $\alpha$) to maximize the cosine similarity between $U_{syn}$ and $U_{real}$. The synthetic images are visualized on the left and resemble noise, preserving user privacy.
• Figure 2: The top row visualizes actual synthetic images ($x_{syn}$) generated by our algorithm. We show $5$ randomly picked images from a set of $32$ images encoding a weight update at the $200^{th}$ communication round of VGG13 on CIFAR-10, Synfreq = $1$ epoch. The bottom row visualizes images recovered with the IG attack ig:geiping2020 on the decoded weight update. Neither sets of images resemble CIFAR-10 images, protecting user data from leakage.
• Figure 3: Synthetic Images produced through the various communication rounds of TOFU. The rows, from top to bottom are, the 32 images encoding the up-communication of Client 1 at the start of the algorithm, at the $200^{th}$ communication round, and at the communication round with maximum accuracy ($593^{rd}$ communication round). This was performed on the CIFAR-10 dataset, distributed across 5 clients in an IID fashion, encoding at the end of every epoch. We see that the images do not bear any resemblance to the dataset.
• Figure 4: Single Device Performance of synthetic images for the MNIST dataset trained on a LeNet5 architecture.
• Figure 5: Single Device Performance of synthetic images for the CIFAR10 dataset trained on a VGG13 architecture.