FeDecider: An LLM-Based Framework for Federated Cross-Domain Recommendation

TL;DR

FeDecider presents an LLM-based framework for federated cross-domain recommendation that splits LoRA updates into directional components and enables per-client data-aware integration through personalized weights. By transmitting only the directional components and letting each client learn weights to combine external directions, FeDecider mitigates scale-related noise and aligns cross-domain knowledge with local distributions. Empirical results on Goodreads and Amazon demonstrate robust improvements over strong baselines, with ablations confirming the necessity of directional decomposition and personalization. The work also analyzes privacy considerations and communication efficiency, showing FeDecider provides a favorable trade-off for scalable, privacy-preserving cross-domain LLM fine-tuning.

Abstract

Federated cross-domain recommendation (Federated CDR) aims to collaboratively learn personalized recommendation models across heterogeneous domains while preserving data privacy. Recently, large language model (LLM)-based recommendation models have demonstrated impressive performance by leveraging LLMs' strong reasoning capabilities and broad knowledge. However, adopting LLM-based recommendation models in Federated CDR scenarios introduces new challenges. First, there exists a risk of overfitting with domain-specific local adapters. The magnitudes of locally optimized parameter updates often vary across domains, causing biased aggregation and overfitting toward domain-specific distributions. Second, unlike traditional recommendation models (e.g., collaborative filtering, bipartite graph-based methods) that learn explicit and comparable user/item representations, LLMs encode knowledge implicitly through autoregressive text generation training. This poses additional challenges for effectively measuring the cross-domain similarities under heterogeneity. To address these challenges, we propose an LLM-based framework for federated cross-domain recommendation, FeDecider. Specifically, FeDecider tackles the challenge of scale-specific noise by disentangling each client's low-rank updates and sharing only their directional components. To handle the need for flexible and effective integration, each client further learns personalized weights that achieve the data-aware integration of updates from other domains. Extensive experiments across diverse datasets validate the effectiveness of our proposed FeDecider.

Figures (7)

• Figure 1: Left: Illustration of the federated cross-domain recommendation. Right: Comparison of performance (Hit@5) between local training and FedAvg aggregation across 3 domains in Goodreads.
• Figure 2: Performance degradation (NDCG@5) of two baseline aggregation strategies across domains in GoodReads dataset, illustrating the challenges of scale-specific noise and ineffective cross-domain similarity measurement with LoRA. The aggregation weights of the baseline LoRA-based Sim are computed using the layer-wise cosine similarity of the LoRA module.
• Figure 3: Overview of FeDecider. Server (middle): At each federated round, the server collects (grey arrow) LoRA updates from all clients, extracts their directional components, and then broadcasts them to all clients (grey arrow). Clients (left and right): Each client (e.g., Domain 1 for books and Domain n for movies) keeps the base model frozen and integrates the received directional components using a set of personalized weights $\alpha$, and only updates its own LoRA and $\alpha$ during local training.
• Figure 4: Visualization of item (left) and user (middle) distances, and learned personalization weights (right, where each row represents the personalization magnitudes learned on a specific client/domain with respect to other domains) across domains (clients).
• Figure 5: Convergence of personalized weights learned on each clients (domains) on GoodReads.

Theorems & Definitions (6)

• definition 1: Harmful and beneficial directions
• lemma 1: First-order expansion
• lemma 2: Representation in the shared subspace
• proposition 1: Personalization capacity
• lemma 3: Gradient w.r.t. personalized weights
• proposition 2: Gradient descent down-weights harmful directions