Privacy-aware Berrut Approximated Coded Computing applied to general distributed learning

TL;DR

Privacy-aware Berrut Approximated Coded Computing (PBACC) extends BACC to provide bounded information leakage in distributed and decentralized learning settings, enabling approximate function evaluation with privacy protection under a threat of up to $c$ honest-but-curious nodes. It generalizes PBACC to tensor inputs and multiple data owners (Generalized PBACC) and integrates it into distributed learning workflows, including secure aggregation and secure training for both centralized and decentralized data. Empirical results on CNNs, VAEs, and Cox regression show learning quality close to non-secure baselines while keeping leakage below a small fraction of a bit per participant; encoding/decoding costs scale with the degree of data decentralization. The work broadens practical privacy-preserving distributed learning by enabling flexible, approximate privacy guarantees for arbitrary functions without strict exact-recovery requirements.

Abstract

Coded computing is one of the techniques that can be used for privacy protection in Federated Learning. However, most of the constructions used for coded computing work only under the assumption that the computations involved are exact, generally restricted to special classes of functions, and require quantized inputs. This paper considers the use of Private Berrut Approximate Coded Computing (PBACC) as a general solution to add strong but non-perfect privacy to federated learning. We derive new adapted PBACC algorithms for centralized aggregation, secure distributed training with centralized data, and secure decentralized training with decentralized data, thus enlarging significantly the applications of the method and the existing privacy protection tools available for these paradigms. Particularly, PBACC can be used robustly to attain privacy guarantees in decentralized federated learning for a variety of models. Our numerical results show that the achievable quality of different learning models (convolutional neural networks, variational autoencoders, and Cox regression) is minimally altered by using these new computing schemes, and that the privacy leakage can be bounded strictly to less than a fraction of one bit per participant. Additionally, the computational cost of the encoding and decoding processes depends only of the degree of decentralization of the data.

Figures (6)

• Figure 1: Distributed training over centralized data
• Figure 2: Secure aggregation over decentralized data
• Figure 3: Secure Training over decentralized data
• Figure 4: Comparison of the Accuracy evolution of all scenarios for the CNN experiments
• Figure 5: Comparison of the Accuracy evolution of all scenarios for the VAE experiments