FedPDD: A Privacy-preserving Double Distillation Framework for Cross-silo Federated Recommendation

TL;DR

This work addresses privacy-preserving cross-silo federated recommendation when overlapped user data is limited. It introduces FedPDD, a double distillation framework that learns from both implicit knowledge (past local predictions) and explicit knowledge (other party via ensemble predictions), while employing an offline training regime and differential privacy to minimize communication and leakage. Empirical results on HetRec-MovieLens and Criteo show FedPDD substantially improves local models and joint predictions relative to state-of-the-art baselines, with gains up to about 3.94–3.98 percentage points. The approach offers a scalable, privacy-conscious solution for cross-silo collaborations with heterogeneous feature spaces and limited data overlap, enabling more robust recommendations without sharing raw data or gradients.

Abstract

Cross-platform recommendation aims to improve recommendation accuracy by gathering heterogeneous features from different platforms. However, such cross-silo collaborations between platforms are restricted by increasingly stringent privacy protection regulations, thus data cannot be aggregated for training. Federated learning (FL) is a practical solution to deal with the data silo problem in recommendation scenarios. Existing cross-silo FL methods transmit model information to collaboratively build a global model by leveraging the data of overlapped users. However, in reality, the number of overlapped users is often very small, thus largely limiting the performance of such approaches. Moreover, transmitting model information during training requires high communication costs and may cause serious privacy leakage. In this paper, we propose a novel privacy-preserving double distillation framework named FedPDD for cross-silo federated recommendation, which efficiently transfers knowledge when overlapped users are limited. Specifically, our double distillation strategy enables local models to learn not only explicit knowledge from the other party but also implicit knowledge from its past predictions. Moreover, to ensure privacy and high efficiency, we employ an offline training scheme to reduce communication needs and privacy leakage risk. In addition, we adopt differential privacy to further protect the transmitted information. The experiments on two real-world recommendation datasets, HetRec-MovieLens and Criteo, demonstrate the effectiveness of FedPDD compared to the state-of-the-art approaches.

Figures (5)

• Figure 1: In cross-device FL setting, participants are a large number of individual customers (2C) that have the same feature space. In cross-silo FL setting, participants are a small number of business partners (2B) that have partially overlapped user spaces and different feature spaces. Here we assume that there is no feature overlapping in our cross-silo FL setting. Data in the red box are used for training.
• Figure 2: The overview of our proposed FedPDD. During training, each party trains its local model via three kinds of knowledge from the ground truth labels, ensemble of local models and past predictions of local models.
• Figure 3: The relationship between communication round $r$ and performance of FedPDD during training
• Figure 4: The comparison between FedPDD and FTL baseline when the overlapped data ratio $\alpha$ decreases
• Figure 5: The impact of DP parameter $\epsilon$ on model performance on HetRec-MovieLens dataset.

Theorems & Definitions (4)

• Definition 1
• Definition 2
• Definition 3
• Definition 4