SCaLRec: Semantic Calibration for LLM-enabled Cloud-Device Sequential Recommendation

TL;DR

The paper tackles semantic staleness in LLM-enabled cloud–device sequential recommendation, where cached cloud semantics may misalign with current user behavior. It introduces SCaLRec, a lightweight on-device framework consisting of a reliability estimator and a semantic calibrator that adjust the cached semantic embedding without invoking cloud LLMs per request, preserving the existing dot-product fusion. The calibrator learns via offline knowledge distillation from fresh semantics to produce embedding-level corrections, which are applied during on-device reranking with a reliability-guided boundary. Empirical results on ReDial and Foursquare show consistent, model-agnostic improvements over strong baselines under varying staleness, underscoring the practical value of maintaining robust, privacy-preserving reranking in intermittent-cloud scenarios. The work highlights a pragmatic path to sustaining high-quality recommendations when cloud access is constrained, by compensating cached semantic signals through calibrated embeddings rather than blanket deprecation of semantic information.

Abstract

Cloud-device collaborative recommendation partitions computation across the cloud and user devices: the cloud provides semantic user modeling, while the device leverages recent interactions and cloud semantic signals for privacy-preserving, responsive reranking. With large language models (LLMs) on the cloud, semantic user representations can improve sequential recommendation by capturing high-level intent. However, regenerating such representations via cloud LLM inference for every request is often infeasible at real-world scale. As a result, on-device reranking commonly reuses a cached cloud semantic user embedding across requests. We empirically identify a cloud semantic staleness effect: reused embeddings become less aligned with the user's latest interactions, leading to measurable ranking degradation. Most existing LLM-enabled cloud-device recommenders are typically designed around on-demand cloud semantics, either by assuming low-latency cloud LLM access or by regenerating semantic embeddings per request. When per-request regeneration is infeasible and cached semantics must be reused, two technical challenges arise: (1) deciding when cached cloud semantics remain useful for on-device reranking, and (2) maintaining ranking quality when the cloud LLM cannot be invoked and only cached semantics are available. To address this gap, we introduce the Semantic Calibration for LLM-enabled Cloud-Device Recommendation (SCaLRec). First, it estimates the reliability of cached semantics under the user's latest interactions. Second, an on-device semantic calibration module is proposed to adjusts the cached semantic embedding on-device using up-to-date interaction evidence, without per-request cloud LLM involvement. Experiments on real-world datasets show that SCaLRec consistently improves recommendation performance over strong baselines under cloud semantic staleness.

Figures (6)

• Figure 1: Cached cloud semantics becomes increasingly harmful as reuse grows. On Foursquare, reranking drops steadily with larger staleness gaps $g$ in both S1 (stale embedding only) and S2 (cached embedding and candidates).
• Figure 2: Overview of the proposed SCaLRec.
• Figure 3: Performance under increasing staleness gap $g$. We compare the SCaLRec under S1 (fixed-candidate reranking) and S2 (end-to-end cached inference). Metrics are NDCG@10 and HR@10 on Foursquare dataset.
• Figure 4: Sensitivity of SCaLRec to correction radius, distillation temperature, and loss weights on Foursquare. Performance (NDCG@10) are reported under S2 settings and averaged over different gaps in $\{5,10,20\}$.
• Figure 5: Model-agnostic integration on Foursquare under semantic staleness. We plug SCaLRec into different on-device structural backbones (SASRec and SURGE) and into a cloud-device framework (CDA4Rec).