PTF-FSR: A Parameter Transmission-Free Federated Sequential Recommender System

TL;DR

A parameter transmission-free federated sequential recommendation framework (PTF-FSR), which ensures both model and data privacy protection to meet the privacy needs of service providers and system users alike and can accommodate more complex and larger sequential recommendation models.

Abstract

Sequential recommender systems have made significant progress. Recently, due to increasing concerns about user data privacy, some researchers have implemented federated learning for sequential recommendation, a.k.a., Federated Sequential Recommender Systems (FedSeqRecs), in which a public sequential recommender model is shared and frequently transmitted between a central server and clients to achieve collaborative learning. Although these solutions mitigate user privacy to some extent, they present two significant limitations that affect their practical usability: (1) They require a globally shared sequential recommendation model. However, in real-world scenarios, the recommendation model constitutes a critical intellectual property for platform and service providers. Therefore, service providers may be reluctant to disclose their meticulously developed models. (2) The communication costs are high as they correlate with the number of model parameters. This becomes particularly problematic as the current FedSeqRec will be inapplicable when sequential recommendation marches into a large language model era. To overcome the above challenges, this paper proposes a parameter transmission-free federated sequential recommendation framework (PTF-FSR), which ensures both model and data privacy protection to meet the privacy needs of service providers and system users alike. Furthermore, since PTF-FSR only transmits prediction results under privacy protection, which are independent of model sizes, this new federated learning architecture can accommodate more complex and larger sequential recommendation models. Extensive experiments conducted on three widely used recommendation datasets, employing various sequential recommendation models from both ID-based and ID-free paradigms, demonstrate the effectiveness and generalization capability of our proposed framework.

Figures (4)

• Figure 1: Traditional parameter transmission-based framework v.s. our parameter transmission-free framework. Traditional framework leverages model parameters to transfer knowledge between clients and the central server, therefore, it suffers model disclosure and heavy communication problems. However, our parameter transmission-free framework replaces the parameters with certain carriers, there, our framework can conceal the model and if the carrier is lightweight, the communication cost will be affordable.
• Figure 2: PTF-FSR includes four steps. Clients first train their client models on local datasets. After that, they utilize the trained client models to generate sequences with the exponential mechanism and send the sequences to the central server. The server trains its delicate model on the noisy data with several contrastive auxiliary tasks. Finally, the central server utilizes the trained server model to return some knowledge back to clients.
• Figure 3: The impact of privacy parameters $\beta$ and $\epsilon$ for (a) PTF-FSR(SASRec) and (b) PTF-FSR(MoRec) on Cell Phone dataset. Similar trends can be observed in the other two datasets. Note that when we investigate one hyperparameter, the other is set to the default value mentioned in Section \ref{['sec_implementation_details']}. That is to say, $\epsilon=1.0$ when $\beta$ changes and $\beta=0.5$ when $\epsilon$ be modified.
• Figure 4: The impact of contrastive factors $\lambda_{pc}$ and $\lambda_{is}$ for (a) PTF-FSR(SASRec) and (b) PTF-FSR(MoRec) on Cell Phone dataset. Similar trends can be observed in the other two datasets. When we investigate one factor, another factor is keeping the default value, i.e., $\lambda_{pc/is}=0.01$.