Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Comparison of Methods for Neural Network Aggregation

John Pomerat, Aviv Segev

TL;DR

This work addresses privacy concerns in joint neural network training across multiple organizations by adopting secure multi-party computation (MPC) to enable aggregation without exposing private data. It formalizes neural network aggregation and examines three methods—series network learning, average ensemble learning, and transfer learning—against a data-sharing baseline in both synthetic regression and a real breast cancer classification task. The results show that series network learning and transfer learning often match or surpass the data-sharing performance, highlighting their potential as privacy-preserving alternatives in healthcare ML; however, membership inference remains a security concern to be mitigated in future studies. The study demonstrates a practical path toward collaborative ML across private datasets with comparable model quality, underscoring both the promise and the challenges of scalable privacy-preserving neural network aggregation.

Abstract

Deep learning has been successful in the theoretical aspect. For deep learning to succeed in industry, we need to have algorithms capable of handling many inconsistencies appearing in real data. These inconsistencies can have large effects on the implementation of a deep learning algorithm. Artificial Intelligence is currently changing the medical industry. However, receiving authorization to use medical data for training machine learning algorithms is a huge hurdle. A possible solution is sharing the data without sharing the patient information. We propose a multi-party computation protocol for the deep learning algorithm. The protocol enables to conserve both the privacy and the security of the training data. Three approaches of neural networks assembly are analyzed: transfer learning, average ensemble learning, and series network learning. The results are compared to approaches based on data-sharing in different experiments. We analyze the security issues of the proposed protocol. Although the analysis is based on medical data, the results of multi-party computation of machine learning training are theoretical and can be implemented in multiple research areas.

A Comparison of Methods for Neural Network Aggregation

TL;DR

This work addresses privacy concerns in joint neural network training across multiple organizations by adopting secure multi-party computation (MPC) to enable aggregation without exposing private data. It formalizes neural network aggregation and examines three methods—series network learning, average ensemble learning, and transfer learning—against a data-sharing baseline in both synthetic regression and a real breast cancer classification task. The results show that series network learning and transfer learning often match or surpass the data-sharing performance, highlighting their potential as privacy-preserving alternatives in healthcare ML; however, membership inference remains a security concern to be mitigated in future studies. The study demonstrates a practical path toward collaborative ML across private datasets with comparable model quality, underscoring both the promise and the challenges of scalable privacy-preserving neural network aggregation.

Abstract

Deep learning has been successful in the theoretical aspect. For deep learning to succeed in industry, we need to have algorithms capable of handling many inconsistencies appearing in real data. These inconsistencies can have large effects on the implementation of a deep learning algorithm. Artificial Intelligence is currently changing the medical industry. However, receiving authorization to use medical data for training machine learning algorithms is a huge hurdle. A possible solution is sharing the data without sharing the patient information. We propose a multi-party computation protocol for the deep learning algorithm. The protocol enables to conserve both the privacy and the security of the training data. Three approaches of neural networks assembly are analyzed: transfer learning, average ensemble learning, and series network learning. The results are compared to approaches based on data-sharing in different experiments. We analyze the security issues of the proposed protocol. Although the analysis is based on medical data, the results of multi-party computation of machine learning training are theoretical and can be implemented in multiple research areas.
Paper Structure (14 sections, 3 equations, 6 figures, 2 tables, 2 algorithms)

This paper contains 14 sections, 3 equations, 6 figures, 2 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Security
  4. Transfer Learning
  5. Neural Network Aggregation
  6. Problem Statement
  7. Series Network Learning
  8. Average Ensemble Learning
  9. Transfer Learning
  10. Experiments
  11. Data
  12. Regression
  13. Breast Cancer Classification
  14. Conclusion

Figures (6)

  • Figure 1: Neural network as an input to assist training a second network.
  • Figure 2: Averaging the corresponding weights and biases of a neural network
  • Figure 3: Random x values (red) and calculated y values (green) on the generated polynomial function (blue).
  • Figure 4: Loss comparison of methods on regression data
  • Figure 5: Accuracy comparison of methods on breast cancer data
  • ...and 1 more figures