Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Row-wise Fusion Regularization: An Interpretable Personalized Federated Learning Framework in Large-Scale Scenarios

Runlin Zhou, Letian Li, Zemin Zheng

TL;DR

The paper addresses personalized federated learning for multivariate outcomes under client heterogeneity and cross-variable dependencies. It introduces Sparse Row-wise Fusion (SROF) to cluster coefficient rows across clients while enforcing within-row sparsity, and develops RowFed, a communication-efficient, partially-participating, linearized-ADMM algorithm. The authors prove an oracle property for SROF, establish asymptotic normality for nonzero components, and show convergence of RowFed to a stationary point with faster iterate-gap contraction under higher participation. Empirically, RowFed consistently improves estimation and prediction accuracy and yields interpretable, variable-level clustering in synthetic studies and a real-world Communities and Crime dataset, outperforming NonFed, FedAvg, and a matrix-fusion baseline. Collectively, row-wise fusion provides a scalable, transparent framework that bridges entry-wise and matrix-wise approaches for large-scale personalized federated multivariate learning.

Abstract

We study personalized federated learning for multivariate responses where client models are heterogeneous yet share variable-level structure. Existing entry-wise penalties ignore cross-response dependence, while matrix-wise fusion over-couples clients. We propose a Sparse Row-wise Fusion (SROF) regularizer that clusters row vectors across clients and induces within-row sparsity, and we develop RowFed, a communication-efficient federated algorithm that embeds SROF into a linearized ADMM framework with privacy-preserving partial participation. Theoretically, we establish an oracle property for SROF-achieving correct variable-level group recovery with asymptotic normality-and prove convergence of RowFed to a stationary solution. Under random client participation, the iterate gap contracts at a rate that improves with participation probability. Empirically, simulations in heterogeneous regimes show that RowFed consistently lowers estimation and prediction error and strengthens variable-level cluster recovery over NonFed, FedAvg, and a personalized matrix-fusion baseline. A real-data study further corroborates these gains while preserving interpretability. Together, our results position row-wise fusion as an effective and transparent paradigm for large-scale personalized federated multivariate learning, bridging the gap between entry-wise and matrix-wise formulations.

Row-wise Fusion Regularization: An Interpretable Personalized Federated Learning Framework in Large-Scale Scenarios

TL;DR

The paper addresses personalized federated learning for multivariate outcomes under client heterogeneity and cross-variable dependencies. It introduces Sparse Row-wise Fusion (SROF) to cluster coefficient rows across clients while enforcing within-row sparsity, and develops RowFed, a communication-efficient, partially-participating, linearized-ADMM algorithm. The authors prove an oracle property for SROF, establish asymptotic normality for nonzero components, and show convergence of RowFed to a stationary point with faster iterate-gap contraction under higher participation. Empirically, RowFed consistently improves estimation and prediction accuracy and yields interpretable, variable-level clustering in synthetic studies and a real-world Communities and Crime dataset, outperforming NonFed, FedAvg, and a matrix-fusion baseline. Collectively, row-wise fusion provides a scalable, transparent framework that bridges entry-wise and matrix-wise approaches for large-scale personalized federated multivariate learning.

Abstract

We study personalized federated learning for multivariate responses where client models are heterogeneous yet share variable-level structure. Existing entry-wise penalties ignore cross-response dependence, while matrix-wise fusion over-couples clients. We propose a Sparse Row-wise Fusion (SROF) regularizer that clusters row vectors across clients and induces within-row sparsity, and we develop RowFed, a communication-efficient federated algorithm that embeds SROF into a linearized ADMM framework with privacy-preserving partial participation. Theoretically, we establish an oracle property for SROF-achieving correct variable-level group recovery with asymptotic normality-and prove convergence of RowFed to a stationary solution. Under random client participation, the iterate gap contracts at a rate that improves with participation probability. Empirically, simulations in heterogeneous regimes show that RowFed consistently lowers estimation and prediction error and strengthens variable-level cluster recovery over NonFed, FedAvg, and a personalized matrix-fusion baseline. A real-data study further corroborates these gains while preserving interpretability. Together, our results position row-wise fusion as an effective and transparent paradigm for large-scale personalized federated multivariate learning, bridging the gap between entry-wise and matrix-wise formulations.
Paper Structure (28 sections, 11 theorems, 101 equations, 2 figures, 3 tables, 1 algorithm)

This paper contains 28 sections, 11 theorems, 101 equations, 2 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work and Our Distinctions
  3. Problem Formulation
  4. Model
  5. Background on Federated Learning in Large-scale Multivariate Scenarios
  6. Row-wise Variable Fusion Analysis: The RowFed Algorithms
  7. Problem reformulation
  8. Detial about RowFed algorithm
  9. RowFed Convergence Analysis
  10. Statistical Consistency: "SROF" Oracle Property
  11. Technical Notation
  12. Theoretical Results
  13. Numerical Evaluation and Real Data Application
  14. Synthetic Data Experiments
  15. Real Data Performance Comparison
  16. ...and 13 more sections

Key Result

Theorem 1

Let assumptions in Lemma 1 and Assumptions 2 hold. If $\rho^{(t+1)}=\alpha \rho^{(t)}$ and $\frac{\min \left\{\lambda_{1}^{2}, \lambda_{2}^{2}\right\}}{2}>\mathcal{L}_{0}+\frac{c_{\lambda} \alpha(\alpha+1)}{2 \rho^{(0)}(\alpha-1)}$ , where $c_{\lambda}$ is a positive constant decided by $\lambda_{1}

Figures (2)

  • Figure 1: Varied dimensions $(p,q)$. Means (s.d.) over 100 replicates. MSE-Est / MSE-Pred scaled by $10^2$.
  • Figure 2: Boxplots of MSEs across states for PerFL-LSMA and RowFed.

Theorems & Definitions (11)

  • Theorem 1: Convergence of Algorithm 1
  • Proposition 1
  • Theorem 2
  • Theorem 3
  • Lemma 1: KKT Conditions
  • Theorem 4
  • Theorem 5
  • Proposition 2: Lipschitz continuous property
  • Lemma 2: Lagrangian Reduction of Updating $\bm\Theta$
  • Lemma 3
  • ...and 1 more