Prototype-Aligned Federated Soft-Prompts for Continual Web Personalization

Abstract

Continual web personalization is essential for engagement, yet real-world non-stationarity and privacy constraints make it hard to adapt quickly without forgetting long-term preferences. We target this gap by seeking a privacy-conscious, parameter-efficient interface that controls stability-plasticity at the user/session level while tying user memory to a shared semantic prior. We propose ProtoFed-SP, a prompt-based framework that injects dual-timescale soft prompts into a frozen backbone: a fast, sparse short-term prompt tracks session intent, while a slow long-term prompt is anchored to a small server-side prototype library that is continually refreshed via differentially private federated aggregation. Queries are routed to Top-M prototypes to compose a personalized prompt. Across eight benchmarks, ProtoFed-SP improves NDCG@10 by +2.9% and HR@10 by +2.0% over the strongest baselines, with notable gains on Amazon-Books (+5.0% NDCG vs. INFER), H&M (+2.5% vs. Dual-LoRA), and Taobao (+2.2% vs. FedRAP). It also lowers forgetting (AF) and Steps-to-95% and preserves accuracy under practical DP budgets. Our contribution is a unifying, privacy-aware prompting interface with prototype anchoring that delivers robust continual personalization and offers a transparent, controllable mechanism to balance stability and plasticity in deployment.

Figures (5)

• Figure 1: ProtoFed-SP: Prototype-Aligned Federated Soft-Prompt Personalization.Left (Client side). Given a user query or interaction history, the client encodes the context into a pooled embedding and routes it to a small set of Top-$M$ population prototypes via similarity-based retrieval. A dual-timescale prompt state is maintained locally: a long-term prompt capturing stable preferences and a short-term prompt modeling session-level intent. The routed prompt is composed by summing the long-term prompt, a drift-adaptive short-term prompt, and a weighted mixture of retrieved prototypes, and is injected into a frozen backbone for ranking. Gradients from the recommendation loss (BCE/BPR) update only the prompt parameters, with sparse proximal updates for the short-term prompt and prototype-aligned updates for the long-term prompt. Right (Server side). Clients periodically upload compressed long-term prompt embeddings protected by clipping and Gaussian noise. The server performs differentially private aggregation (e.g., DP-FedKMeans or robust alternatives) to update a compact prototype library, with momentum updates, prototype separation, and pruning to prevent collapse. The refreshed prototypes are broadcast back to clients, enabling privacy-conscious, parameter-efficient continual personalization with explicit stability--plasticity control.
• Figure 2: Core hyperparameter sensitivity (NDCG@10, mean$\pm$std). One parameter varied per block; others fixed at defaults.
• Figure 3: Stability–plasticity verification. Prototype alignment consistently lowers forgetting, while short-term prompts markedly reduce the steps needed to adapt; combining both yields the best AF–NDCG frontier.
• Figure 4: Privacy–utility trade-off. Across datasets, moderate DP (e.g., $\sigma\!=\!0.4$, mid $\varepsilon$) preserves personalization: NDCG stays within the Non-DP band while AF increases only slightly; very strong DP ($\sigma\!=\!0.8$) degrades gracefully rather than catastrophically.
• Figure 5: High-drift users see the largest, most significant gains; cold-start users also benefit. Heavy users gain, but improvements are not exclusive to them.

Theorems & Definitions (4)

• Remark 1: Relation to ProtoFed-SP updates
• lemma 1: Proximal three-point inequality
• theorem 1: Dynamic regret bound (path-length form)
• proposition 1: Anchor-induced contraction of the optimal path