Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Parallel Collaborative ADMM Privacy Computing and Adaptive GPU Acceleration for Distributed Edge Networks

Mengchun Xia, Zhicheng Dong, Donghong Cai, Fang Fang, Lisheng Fan, Pingzhi Fan

TL;DR

The work tackles privacy-preserving distributed optimization in edge networks by integrating parallel ADMM with Paillier homomorphic encryption. It introduces 3P-ADMM-PC2, a three-phase framework that uses a novel floating-point quantization scheme to enable homomorphic operations and GPU-based acceleration to manage large key spaces via CRT-based Decomposition. The approach achieves mean-square-error performance near that of privacy-free distributed ADMM while delivering substantial speedups over CPU-based privacy schemes, as demonstrated on large-scale sparse recovery and power-network reconstruction tasks. By distributing encryption/decryption across edge nodes and exploiting GPU parallelism, the method provides scalable, privacy-preserving collaboration suitable for real-time edge computing applications.

Abstract

Distributed computing has been widely applied in distributed edge networks for reducing the processing burden of high-dimensional data centralization, where a high-dimensional computational task is decomposed into multiple low-dimensional collaborative processing tasks or multiple edge nodes use distributed data to train a global model. However, the computing power of a single-edge node is limited, and collaborative computing will cause information leakage and excessive communication overhead. In this paper, we design a parallel collaborative distributed alternating direction method of multipliers (ADMM) and propose a three-phase parallel collaborative ADMM privacy computing (3P-ADMM-PC2) algorithm for distributed computing in edge networks, where the Paillier homomorphic encryption is utilized to protect data privacy during interactions. Especially, a quantization method is introduced, which maps the real numbers to a positive integer interval without affecting the homomorphic operations. To address the architectural mismatch between large-integer and Graphics Processing Unit (GPU) computing, we transform high-bitwidth computations into low-bitwidth matrix and vector operations. Thus the GPU can be utilized to implement parallel encryption and decryption computations with long keys. Finally, a GPU-accelerated 3P-ADMM-PC2 is proposed to optimize the collaborative computing tasks. Meanwhile, large-scale computational tasks are conducted in network topologies with varying numbers of edge nodes. Experimental results demonstrate that the proposed 3P-ADMM-PC2 has excellent mean square error performance, which is close to that of distributed ADMM without privacy-preserving. Compared to centralized ADMM and distributed ADMM implemented with Central Processing Unit (CPU) computation, the proposed scheme demonstrates a significant speedup ratio.

Parallel Collaborative ADMM Privacy Computing and Adaptive GPU Acceleration for Distributed Edge Networks

TL;DR

The work tackles privacy-preserving distributed optimization in edge networks by integrating parallel ADMM with Paillier homomorphic encryption. It introduces 3P-ADMM-PC2, a three-phase framework that uses a novel floating-point quantization scheme to enable homomorphic operations and GPU-based acceleration to manage large key spaces via CRT-based Decomposition. The approach achieves mean-square-error performance near that of privacy-free distributed ADMM while delivering substantial speedups over CPU-based privacy schemes, as demonstrated on large-scale sparse recovery and power-network reconstruction tasks. By distributing encryption/decryption across edge nodes and exploiting GPU parallelism, the method provides scalable, privacy-preserving collaboration suitable for real-time edge computing applications.

Abstract

Distributed computing has been widely applied in distributed edge networks for reducing the processing burden of high-dimensional data centralization, where a high-dimensional computational task is decomposed into multiple low-dimensional collaborative processing tasks or multiple edge nodes use distributed data to train a global model. However, the computing power of a single-edge node is limited, and collaborative computing will cause information leakage and excessive communication overhead. In this paper, we design a parallel collaborative distributed alternating direction method of multipliers (ADMM) and propose a three-phase parallel collaborative ADMM privacy computing (3P-ADMM-PC2) algorithm for distributed computing in edge networks, where the Paillier homomorphic encryption is utilized to protect data privacy during interactions. Especially, a quantization method is introduced, which maps the real numbers to a positive integer interval without affecting the homomorphic operations. To address the architectural mismatch between large-integer and Graphics Processing Unit (GPU) computing, we transform high-bitwidth computations into low-bitwidth matrix and vector operations. Thus the GPU can be utilized to implement parallel encryption and decryption computations with long keys. Finally, a GPU-accelerated 3P-ADMM-PC2 is proposed to optimize the collaborative computing tasks. Meanwhile, large-scale computational tasks are conducted in network topologies with varying numbers of edge nodes. Experimental results demonstrate that the proposed 3P-ADMM-PC2 has excellent mean square error performance, which is close to that of distributed ADMM without privacy-preserving. Compared to centralized ADMM and distributed ADMM implemented with Central Processing Unit (CPU) computation, the proposed scheme demonstrates a significant speedup ratio.
Paper Structure (24 sections, 3 theorems, 45 equations, 10 figures, 5 tables, 3 algorithms)

This paper contains 24 sections, 3 theorems, 45 equations, 10 figures, 5 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. System Model and Problem Formulation
  3. Parallel Collaborative ADMM Privacy Computing
  4. Quantization of Floating Point Type Data
  5. Encryption and Distributed ADMM Privacy Computation
  6. Decryption and Inverse Quantization
  7. Adaptive GPU Accelerated 3P-ADMM-PC2
  8. Parallel Computing in Streaming Multiprocessor
  9. Multiplication of Large Integers
  10. ModExp Computing Task Decomposition
  11. Modular Operation
  12. Proposed GPU-accelerated 3P-ADMM-PC2
  13. The First Communication Round
  14. The Second Communication Round
  15. The Third Communication Round
  16. ...and 9 more sections

Key Result

Theorem 1

For quantization $\Gamma_1, \Gamma_2$, and $\mathbf{u}_1, \mathbf{u}_2, \mathbf{u}_{3}\in\mathbb{R}^N, \mathbf{B}\in\mathbb{R}^{N\times N}$, the quantization calculation in ret54 can be given by which can be further approximate by

Figures (10)

  • Figure 1: Proposed 3P-ADMM-PC2 scheme.
  • Figure 2: GPU-accelerated ModExp computation in large key space.
  • Figure 3: The illustration of proposed GPU-accelerated 3P-ADMM-PC2.
  • Figure 4: Hardware configuration and network environment for distributed computing.
  • Figure 5: Precision loss of the quantization scheme with different $\Delta$ values.
  • ...and 5 more figures

Theorems & Definitions (10)

  • Remark 1
  • Definition 1
  • Definition 2
  • Theorem 1
  • proof
  • Remark 2
  • Lemma 1
  • Lemma 2
  • Remark 3
  • Remark 4