Additive-Effect Assisted Learning

TL;DR

AE-AL addresses decentralized learning when partners hold undisclosed covariates and communication is costly. It introduces a two-stage framework: a privacy-preserving screening stage using a Wald-type test with sketchy data, followed by an iterative training stage where two parties exchange linear combinations of covariates to obtain an oracle-equivalent model with exponentially fast convergence in transmission rounds. The approach applies to generalized linear models under standard regularity conditions, with theoretical guarantees for the screening statistic and the asymptotic normality of the AE-AL estimator, and it demonstrates superior convergence speed and robustness in simulations and real-data tasks. The practical impact lies in enabling privacy-aware, communication-efficient collaboration across institutions or devices while retaining near-centralized predictive performance.

Abstract

It is quite popular nowadays for researchers and data analysts holding different datasets to seek assistance from each other to enhance their modeling performance. We consider a scenario where different learners hold datasets with potentially distinct variables, and their observations can be aligned by a nonprivate identifier. Their collaboration faces the following difficulties: First, learners may need to keep data values or even variable names undisclosed due to, e.g., commercial interest or privacy regulations; second, there are restrictions on the number of transmission rounds between them due to e.g., communication costs. To address these challenges, we develop a two-stage assisted learning architecture for an agent, Alice, to seek assistance from another agent, Bob. In the first stage, we propose a privacy-aware hypothesis testing-based screening method for Alice to decide on the usefulness of the data from Bob, in a way that only requires Bob to transmit sketchy data. Once Alice recognizes Bob's usefulness, Alice and Bob move to the second stage, where they jointly apply a synergistic iterative model training procedure. With limited transmissions of summary statistics, we show that Alice can achieve the oracle performance as if the training were from centralized data, both theoretically and numerically.

Figures (54)

• Figure 1: Illustration of data held by two learners after removing unmatched cases, where the rows(cases) are collated according to a common identifier, e.g., timestamp, username, or case ID.
• Figure 2: An illustration of the data held by a and b.
• Figure 3: A graphical summary of the additive-effect assisted learning (AE-AL).
• Figure 4: The Q-Q plot of $W_{n,t}$ from Setting 2 with AR(1) covariates, $\rho = 0.1$, and the scale of the Laplace noise is 0.5. The red straight line stands for the perfect match between the sample quantiles of the p-values and the theoretical quantiles of $\textrm{Uniform}(0,1)$.
• Figure 5: The rejection rates of $W_{n,t}$ under $\textrm{H}_1$ versus $t$ (the number of columns in the sketchy data) from 100 replications, where $\textrm{Lap}(0)$, $\textrm{Lap}(0.1)$, and $\textrm{Lap}(0.5)$ stand for the sketchy data without noise, with Laplace noise of scale 0.1, and with Laplace noise of scale 0.5, respectively.

Theorems & Definitions (50)

• Theorem 1
• Proposition 1
• Theorem 2
• Corollary 1
• Corollary 2
• Lemma 1: Continuity of eigenvalues
• Proof 1
• Lemma 2: Strong convexity of $M_n(\bm{\beta})$
• Proof 2
• Lemma 3: Zero partial derivatives