Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Efficient Distributed Learning over Decentralized Networks with Convoluted Support Vector Machine

Canyi Chen, Nan Qiao, Liping Zhu

TL;DR

This work tackles efficient sparse classification over decentralized networks using elastic-net penalized SVMs with hinge loss. By replacing the nonsmooth hinge loss with a convolution-type smoothing $L_h$, the authors obtain a convex, smooth objective and develop a generalized ADMM that converges linearly. They prove a smoothing bias of $O(h^2)$ and, with a carefully chosen bandwidth and penalties, achieve near-minimax estimation rates and exact support recovery under a beta-min condition; the convergence rate depends on network topology through a contraction factor. Empirical results on simulated and real data show that deCSVM yields accurate, sparse classifiers with low communication overhead, outperforming several decentralized baselines and demonstrating robustness to topology and label noise. Overall, the approach provides a practical, theoretically justified framework for efficient distributed learning in privacy- and resource-constrained networked environments.

Abstract

This paper addresses the problem of efficiently classifying high-dimensional data over decentralized networks. Penalized support vector machines (SVMs) are widely used for high-dimensional classification tasks. However, the double nonsmoothness of the objective function poses significant challenges in developing efficient decentralized learning methods. Many existing procedures suffer from slow, sublinear convergence rates. To overcome this limitation, we consider a convolution-based smoothing technique for the nonsmooth hinge loss function. The resulting loss function remains convex and smooth. We then develop an efficient generalized alternating direction method of multipliers (ADMM) algorithm for solving penalized SVM over decentralized networks. Our theoretical contributions are twofold. First, we establish that our generalized ADMM algorithm achieves provable linear convergence with a simple implementation. Second, after a sufficient number of ADMM iterations, the final sparse estimator attains near-optimal statistical convergence and accurately recovers the true support of the underlying parameters. Extensive numerical experiments on both simulated and real-world datasets validate our theoretical findings.

Efficient Distributed Learning over Decentralized Networks with Convoluted Support Vector Machine

TL;DR

This work tackles efficient sparse classification over decentralized networks using elastic-net penalized SVMs with hinge loss. By replacing the nonsmooth hinge loss with a convolution-type smoothing , the authors obtain a convex, smooth objective and develop a generalized ADMM that converges linearly. They prove a smoothing bias of and, with a carefully chosen bandwidth and penalties, achieve near-minimax estimation rates and exact support recovery under a beta-min condition; the convergence rate depends on network topology through a contraction factor. Empirical results on simulated and real data show that deCSVM yields accurate, sparse classifiers with low communication overhead, outperforming several decentralized baselines and demonstrating robustness to topology and label noise. Overall, the approach provides a practical, theoretically justified framework for efficient distributed learning in privacy- and resource-constrained networked environments.

Abstract

This paper addresses the problem of efficiently classifying high-dimensional data over decentralized networks. Penalized support vector machines (SVMs) are widely used for high-dimensional classification tasks. However, the double nonsmoothness of the objective function poses significant challenges in developing efficient decentralized learning methods. Many existing procedures suffer from slow, sublinear convergence rates. To overcome this limitation, we consider a convolution-based smoothing technique for the nonsmooth hinge loss function. The resulting loss function remains convex and smooth. We then develop an efficient generalized alternating direction method of multipliers (ADMM) algorithm for solving penalized SVM over decentralized networks. Our theoretical contributions are twofold. First, we establish that our generalized ADMM algorithm achieves provable linear convergence with a simple implementation. Second, after a sufficient number of ADMM iterations, the final sparse estimator attains near-optimal statistical convergence and accurately recovers the true support of the underlying parameters. Extensive numerical experiments on both simulated and real-world datasets validate our theoretical findings.

Paper Structure

This paper contains 16 sections, 15 equations, 2 figures, 6 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Works
  3. Paper Organization and Notations
  4. Methodology
  5. Model Setup
  6. A Convolution-type Smoothing Procedure
  7. Generalized ADMM
  8. Theoretical Properties
  9. Simulation Studies
  10. Design of the Simulations
  11. Effect of Iterations
  12. Effect of Local Sample Size
  13. Effect of Topology
  14. Effect of Sign Flips
  15. An Application
  16. ...and 1 more sections

Figures (2)

  • Figure 1: The horizontal axis stands for the number of iterations, and the vertical axis represents the estimation error of the deCSVM estimate under different kernel types: uniform (), Laplacian (), logistic (), Gaussian (), and Epanechnikov ().
  • Figure 2: Decentralized network of the communities and crime dataset.