Coarse-to-Fine Lightweight Meta-Embedding for ID-Based Recommendation

TL;DR

The paper tackles memory constrained ID_based recommendations by addressing the gap where coarse embeddings fail to capture fine_grained semantics. It introduces a coarse_to_fine meta_embedding framework built on a graph neural network with two tiers of virtual nodes, along with SparsePCA_init and soft_thresholding to enforce sparsity, and a weight_bridging mechanism to align coarse and fine embeddings. Empirical results on three benchmarks show strong improvements in accuracy with reduced parameter counts, supported by ablation and hyperparameter analyses that highlight the value of initialization, sparsity control, and bridging. The approach enables more expressive, memory_efficient representations suitable for on_device or resource_limited deployments, and comes with code availability for reproducibility.

Abstract

The state-of-the-art recommendation systems have shifted the attention to efficient recommendation, e.g., on-device recommendation, under memory constraints. To this end, the existing methods either focused on the lightweight embeddings for both users and items, or involved on-device systems enjoying the compact embeddings to enhance reusability and reduces space complexity. However, they focus solely on the coarse granularity of embedding, while overlook the fine-grained semantic nuances, to adversarially downgrade the efficacy of meta-embeddings in capturing the intricate relationship over both user and item, consequently resulting into the suboptimal recommendations. In this paper, we aim to study how the meta-embedding can efficiently learn varied grained semantics, together with how the fine-grained meta-embedding can strengthen the representation of coarse-grained meta-embedding. To answer these questions, we develop a novel graph neural networks (GNNs) based recommender where each user and item serves as the node, linked directly to coarse-grained virtual nodes and indirectly to fine-grained virtual nodes, ensuring different grained semantic learning, while disclosing: 1) In contrast to coarse-grained semantics, fine-grained semantics are well captured through sparse meta-embeddings, which adaptively 2) balance the embedding uniqueness and memory constraint. Additionally, the initialization method come up upon SparsePCA, along with a soft thresholding activation function to render the sparseness of the meta-embeddings. We propose a weight bridging update strategy that focuses on matching each coarse-grained meta-embedding with several fine-grained meta-embeddings based on the users/items' semantics. Extensive experiments substantiate our method's superiority over existing baselines. Our code is available at https://github.com/htyjers/C2F-MetaEmbed.

Figures (6)

• Figure 1: Illustration of the proposed pipeline. Our basic idea is to construct a the hierarchical and selective connectivity graph (a), which ensures that the model learns wide-ranging semantic information at the coarse level (b), while also focusing on granular, personalized features at the fine level (c)
• Figure 2: Illustration of fine meta embedding initialization strategy, which preserves sparsity and maintains associative relationships between users and items
• Figure 3: Illustration of weight bridging updating strategy for $S^{r}$, which dynamically aligns coarse meta-embeddings with multiple fine meta-embeddings based on users/items' semantic relevance.
• Figure 4: The performance of our method on w.r.t. various model components.
• Figure 5: The performance of our method on w.r.t. various hyperparameter settings.