Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Personalized Federated Collaborative Filtering: A Variational AutoEncoder Approach

Zhiwei Li, Guodong Long, Tianyi Zhou, Jing Jiang, Chengqi Zhang

TL;DR

This paper tackles privacy-preserving federated collaborative filtering by reformulating FedCF as a variational autoencoder task. It introduces FedDAE, a gating dual-encoder VAE where a global encoder captures shared knowledge and a local encoder preserves user-specific information, with a gating network adaptively balancing their contributions. The model optimizes a beta-ELBO with a global decoder and uses additive personalization to achieve strong performance on four real-world, implicit-feedback datasets, approaching centralized methods while maintaining privacy. Findings show that adaptive weighting and dual-encoder design yield superior personalization and generalization, with robustness to privacy-preserving perturbations and dataset sparsity implications highlighted for future work.

Abstract

Federated Collaborative Filtering (FedCF) is an emerging field focused on developing a new recommendation framework with preserving privacy in a federated setting. Existing FedCF methods typically combine distributed Collaborative Filtering (CF) algorithms with privacy-preserving mechanisms, and then preserve personalized information into a user embedding vector. However, the user embedding is usually insufficient to preserve the rich information of the fine-grained personalization across heterogeneous clients. This paper proposes a novel personalized FedCF method by preserving users' personalized information into a latent variable and a neural model simultaneously. Specifically, we decompose the modeling of user knowledge into two encoders, each designed to capture shared knowledge and personalized knowledge separately. A personalized gating network is then applied to balance personalization and generalization between the global and local encoders. Moreover, to effectively train the proposed framework, we model the CF problem as a specialized Variational AutoEncoder (VAE) task by integrating user interaction vector reconstruction with missing value prediction. The decoder is trained to reconstruct the implicit feedback from items the user has interacted with, while also predicting items the user might be interested in but has not yet interacted with. Experimental results on benchmark datasets demonstrate that the proposed method outperforms other baseline methods, showcasing superior performance. Our code is available at https://github.com/mtics/FedDAE.

Personalized Federated Collaborative Filtering: A Variational AutoEncoder Approach

TL;DR

This paper tackles privacy-preserving federated collaborative filtering by reformulating FedCF as a variational autoencoder task. It introduces FedDAE, a gating dual-encoder VAE where a global encoder captures shared knowledge and a local encoder preserves user-specific information, with a gating network adaptively balancing their contributions. The model optimizes a beta-ELBO with a global decoder and uses additive personalization to achieve strong performance on four real-world, implicit-feedback datasets, approaching centralized methods while maintaining privacy. Findings show that adaptive weighting and dual-encoder design yield superior personalization and generalization, with robustness to privacy-preserving perturbations and dataset sparsity implications highlighted for future work.

Abstract

Federated Collaborative Filtering (FedCF) is an emerging field focused on developing a new recommendation framework with preserving privacy in a federated setting. Existing FedCF methods typically combine distributed Collaborative Filtering (CF) algorithms with privacy-preserving mechanisms, and then preserve personalized information into a user embedding vector. However, the user embedding is usually insufficient to preserve the rich information of the fine-grained personalization across heterogeneous clients. This paper proposes a novel personalized FedCF method by preserving users' personalized information into a latent variable and a neural model simultaneously. Specifically, we decompose the modeling of user knowledge into two encoders, each designed to capture shared knowledge and personalized knowledge separately. A personalized gating network is then applied to balance personalization and generalization between the global and local encoders. Moreover, to effectively train the proposed framework, we model the CF problem as a specialized Variational AutoEncoder (VAE) task by integrating user interaction vector reconstruction with missing value prediction. The decoder is trained to reconstruct the implicit feedback from items the user has interacted with, while also predicting items the user might be interested in but has not yet interacted with. Experimental results on benchmark datasets demonstrate that the proposed method outperforms other baseline methods, showcasing superior performance. Our code is available at https://github.com/mtics/FedDAE.
Paper Structure (30 sections, 1 theorem, 7 equations, 3 figures, 4 tables, 1 algorithm)

This paper contains 30 sections, 1 theorem, 7 equations, 3 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Personalized Federated Learning.
  4. Federated Collaborative Filtering.
  5. Variational Autoencoders.
  6. Problem Formulation
  7. Federated Collaborative Filtering (FedCF)
  8. A VAE perspective for FedCF.
  9. Methodology
  10. Framework
  11. Dual Encoders
  12. Gating Network
  13. Reconstruction and Prediction
  14. Algorithm
  15. Discussions
  16. ...and 15 more sections

Key Result

Lemma 1

Given two independent Gaussian random variables $X$ and $Y$, distributed as $N(\mu_1, \sigma_1^2)$ and $N(\mu_2, \sigma_2^2)$, respectively. Then $Z = w_1 X + w_2 Y$ follows a new Gaussian distribution $Z \sim N(w_1 \mu_1 + w_2 \mu_2, w_1^2 \sigma_1^2 + w_2^2 \sigma_2^2)$.

Figures (3)

  • Figure 1: The framework of FedDAE.
  • Figure 2: The visualization of FedDAE's convergence and efficiency across the four used datasets.
  • Figure 3: The t-SNE visualization of item features learned by FedDAE on the ML-100K dataset illustrates the representations across different users. In the visualization, red indicates items that users have interacted with, while blue indicates items they have not interacted with. The global representation is generated by FedDAE's global encoder, and the local representation is produced by the user-specific encoder. Average Combination refers to the simple average of the global and local representations, while Weighted Combination reflects the weighted combination based on the gating network outputs tailored to the client data. FedDAE's adaptive personalization enhances its ability to distinguish between items users have interacted with and those they haven't, leading to improved recommendation performance.

Theorems & Definitions (1)

  • Lemma 1: Additivity of Gaussian distributions