Active-Passive Federated Learning for Vertically Partitioned Multi-view Data

TL;DR

The paper tackles unreliable collaboration in vertical federated learning for vertically partitioned multi-view data by introducing Active-Passive Federated Learning (APFed), where one active client trains a full model with passive clients only assisting during training, enabling independent inference thereafter. APFed formalizes a loss decomposition $L = L_A + \sum_{p=1}^M \lambda_p L_p$ and demonstrates how gradients from passive clients integrate to update the active encoder and the shared task module. Two concrete instantiations, APFed-R (reconstruction loss) and APFed-C (contrastive loss), are proposed and evaluated on MNIST, FMNIST, CIFAR10, and CIFAR100 with multi-view splits, outperforming vertical FL baselines and single-view alternatives. The results show robustness to unpredictable collaboration and provide guidance on selecting the trade-off parameter $\lambda$, highlighting practical deployment implications for privacy-preserving, multi-view learning.

Abstract

Vertical federated learning is a natural and elegant approach to integrate multi-view data vertically partitioned across devices (clients) while preserving their privacies. Apart from the model training, existing methods requires the collaboration of all clients in the model inference. However, the model inference is probably maintained for service in a long time, while the collaboration, especially when the clients belong to different organizations, is unpredictable in real-world scenarios, such as concellation of contract, network unavailablity, etc., resulting in the failure of them. To address this issue, we, at the first attempt, propose a flexible Active-Passive Federated learning (APFed) framework. Specifically, the active client is the initiator of a learning task and responsible to build the complete model, while the passive clients only serve as assistants. Once the model built, the active client can make inference independently. In addition, we instance the APFed framework into two classification methods with employing the reconstruction loss and the contrastive loss on passive clients, respectively. Meanwhile, the two methods are tested in a set of experiments and achieves desired results, validating their effectiveness.

Figures (4)

• Figure 1: Frameworks of typical vertical federated learning and the proposed APFed learning. For brevity, only two clients (the left), two passive clients (the right) and the classification task (the both sides) are presented, respectively. Meanwhile, Graph Convolutional Networks (GCN) and Deep Neural Networks (DNN) are employed to generated data representations. Actually, the client number, task type and representation generation method are not limited. Nevertheless, the arrows across clients and server refer to network transmission. Note that, the two data representations of the left are different (with different colors), while the three of the right are the same (with the same color).
• Figure 2: The $m$-$i$ splitting strategy on MNIST dataset. In the 2-1 setting, an "8" image is cut into 3 parts (views) with deploying the 1-st view into the active client and the 2-nd view to the passive client. Similar is the 3-1 setting.
• Figure 3: The accuracy comparison of the proposed APFed-R method with trade-off $\lambda$ ranging from 0 to 10 in the 2-1 setting.
• Figure 4: The accuracy comparison of the proposed APFed-C method with trade-off $\lambda$ ranging from 0 to 10 in the 2-1 setting.