Personalized Subgraph Federated Learning with Differentiable Auxiliary Projections

TL;DR

This work tackles non-IID challenges in federated learning on graph-structured data by introducing FedAux, a personalized subgraph FL framework that jointly learns a GNN and a differentiable Auxiliary Projection Vector (APV) to map node embeddings into a $1$-D space. Local training employs a continuous kernel aggregation that replaces hard sorting, enabling smooth optimization of the APV, which serves as a compact client signature. The server uses APV-based similarities to perform personalized aggregation, yielding models tailored to each client while preserving cross-client knowledge transfer, with theoretical convergence guarantees and empirical superiority across six graph benchmarks. The APV also demonstrates privacy protection against membership inference attacks and demonstrates transferability to other baselines, making FedAux a scalable, privacy-preserving approach to personalized graph FL.

Abstract

Federated Learning (FL) on graph-structured data typically faces non-IID challenges, particularly in scenarios where each client holds a distinct subgraph sampled from a global graph. In this paper, we introduce Federated learning with Auxiliary projections (FedAux), a personalized subgraph FL framework that learns to align, compare, and aggregate heterogeneously distributed local models without sharing raw data or node embeddings. In FedAux, each client jointly trains (i) a local GNN and (ii) a learnable auxiliary projection vector (APV) that differentiably projects node embeddings onto a 1D space. A soft-sorting operation followed by a lightweight 1D convolution refines these embeddings in the ordered space, enabling the APV to effectively capture client-specific information. After local training, these APVs serve as compact signatures that the server uses to compute inter-client similarities and perform similarity-weighted parameter mixing, yielding personalized models while preserving cross-client knowledge transfer. Moreover, we provide rigorous theoretical analysis to establish the convergence and rationality of our design. Empirical evaluations across diverse graph benchmarks demonstrate that FedAux substantially outperforms existing baselines in both accuracy and personalization performance. The code is available at https://github.com/JhuoW/FedAux.

Figures (12)

• Figure 1: The overall framework of FedAux. Left: The server maintains a global GNN model together with learnable auxiliary projection vectors (APVs) that are broadcast to all clients at the start of each communication round. Middle: Clients jointly optimize the GNN and APV during local training, where the APV projects node embeddings onto a 1D $\boldsymbol{a}_k$-space that positions related nodes closer together. Right: After local training, clients transmit their optimized GNN parameters and personalized APVs to the server. The server computes a client similarity matrix by comparing the learned APVs, which captures the heterogeneity across subgraphs without accessing raw data. These similarities determine personalized aggregation weights. At the end of each round, the updated global parameters are broadcast for the next communication round.
• Figure 2: The local training of FedAux aims to map all nodes in $G_k$ to a corresponding $\boldsymbol{a}_k$-space, and the optimization objective is to learning the APV$\boldsymbol{a}_k$, such that the resulting $\boldsymbol{a}_k$-space preserves the optimal node sorting.
• Figure 3: Client similarity based on different measures. Darker colors indicate higher similarity.
• Figure 4: Transferability of APV.
• Figure 5: Cora with 10 clients

Theorems & Definitions (17)

• Theorem 3.1: Fidelity of the APV $\boldsymbol{a}$
• Theorem 3.2: Sorting limit and equivalence to $\mathrm{Conv1D}$
• Theorem 3.3: Global linear convergence
• Lemma C.3: Gradient of similarity score
• proof
• Lemma C.4: Gradient of kernel entry
• proof
• Lemma C.5: Gradient of the kernel‑smoothed embedding
• proof
• proof