Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Decentralized Smoothing ADMM for Quantile Regression with Non-Convex Sparse Penalties

Reza Mirzaeifard, Diyako Ghaderyan, Stefan Werner

TL;DR

The paper tackles decentralized penalized quantile regression for distributed IoT data using non-convex sparsity penalties. It introduces DSAD, a smoothing ADMM framework that implements a total-variation consensus term and proximal updates for MCP/SCAD penalties, enabling reliable convergence in a multi-node setting. The authors prove convergence to stationary points under weak convexity and demonstrate via simulations that DSAD achieves lower $MSE$ and higher coefficient-recognition accuracy than baselines, with exact sparsity for non-active coefficients. This work offers a scalable, privacy-conscious approach for robust, sparse quantile modeling in distributed sensor networks, where traditional methods struggle with non-convexity and consensus.

Abstract

In the rapidly evolving internet-of-things (IoT) ecosystem, effective data analysis techniques are crucial for handling distributed data generated by sensors. Addressing the limitations of existing methods, such as the sub-gradient approach, which fails to distinguish between active and non-active coefficients effectively, this paper introduces the decentralized smoothing alternating direction method of multipliers (DSAD) for penalized quantile regression. Our method leverages non-convex sparse penalties like the minimax concave penalty (MCP) and smoothly clipped absolute deviation (SCAD), improving the identification and retention of significant predictors. DSAD incorporates a total variation norm within a smoothing ADMM framework, achieving consensus among distributed nodes and ensuring uniform model performance across disparate data sources. This approach overcomes traditional convergence challenges associated with non-convex penalties in decentralized settings. We present theoretical proofs and extensive simulation results to validate the effectiveness of the DSAD, demonstrating its superiority in achieving reliable convergence and enhancing estimation accuracy compared with prior methods.

Decentralized Smoothing ADMM for Quantile Regression with Non-Convex Sparse Penalties

TL;DR

The paper tackles decentralized penalized quantile regression for distributed IoT data using non-convex sparsity penalties. It introduces DSAD, a smoothing ADMM framework that implements a total-variation consensus term and proximal updates for MCP/SCAD penalties, enabling reliable convergence in a multi-node setting. The authors prove convergence to stationary points under weak convexity and demonstrate via simulations that DSAD achieves lower and higher coefficient-recognition accuracy than baselines, with exact sparsity for non-active coefficients. This work offers a scalable, privacy-conscious approach for robust, sparse quantile modeling in distributed sensor networks, where traditional methods struggle with non-convexity and consensus.

Abstract

In the rapidly evolving internet-of-things (IoT) ecosystem, effective data analysis techniques are crucial for handling distributed data generated by sensors. Addressing the limitations of existing methods, such as the sub-gradient approach, which fails to distinguish between active and non-active coefficients effectively, this paper introduces the decentralized smoothing alternating direction method of multipliers (DSAD) for penalized quantile regression. Our method leverages non-convex sparse penalties like the minimax concave penalty (MCP) and smoothly clipped absolute deviation (SCAD), improving the identification and retention of significant predictors. DSAD incorporates a total variation norm within a smoothing ADMM framework, achieving consensus among distributed nodes and ensuring uniform model performance across disparate data sources. This approach overcomes traditional convergence challenges associated with non-convex penalties in decentralized settings. We present theoretical proofs and extensive simulation results to validate the effectiveness of the DSAD, demonstrating its superiority in achieving reliable convergence and enhancing estimation accuracy compared with prior methods.
Paper Structure (5 sections, 1 theorem, 23 equations, 1 figure, 1 algorithm)

This paper contains 5 sections, 1 theorem, 23 equations, 1 figure, 1 algorithm.

Table of Contents

  1. Introduction
  2. Problem formulation
  3. Decentralized Smoothing ADMM
  4. Simulation Results
  5. Conclusion

Key Result

Theorem 1

Let $K$ be a constant such that for every $k \geq K$, we have $\sigma^{(k+1)}_{\xi}>\frac{M \rho}{\min_{p,l} \|\mathbf{X}^{(l)}_{:,p}\|^2_2}$, where $\rho \geq \frac{1}{\gamma}$ for MCP and $\rho \geq \frac{1}{\gamma-1}$ for SCAD represent their weakly convexity conditions. Assume each node has $M$

Figures (1)

  • Figure 1: Performance comparison of DSAD, DSPQ, and $l_1$-dQR

Theorems & Definitions (2)

  • Theorem 1: Global Convergence
  • proof