Lightweight Embeddings for Graph Collaborative Filtering

TL;DR

This work proposes Lightweight Embeddings for Graph Collaborative Filtering (LEGCF), a parameter-efficient embedding framework dedicated to GNN-based recommenders, which innovatively introduces an assignment matrix as an additional learnable component on top of meta-embeddings.

Abstract

Graph neural networks (GNNs) are currently one of the most performant collaborative filtering methods. Meanwhile, owing to the use of an embedding table to represent each user/item as a distinct vector, GNN-based recommenders have inherited the long-standing defect of parameter inefficiency. As a common practice for scalable embeddings, parameter sharing enables the use of fewer embedding vectors (i.e., meta-embeddings). When assigning meta-embeddings, most existing methods are a heuristically designed, predefined mapping from each user's/item's ID to the corresponding meta-embedding indexes, thus simplifying the optimization problem into learning only the meta-embeddings. However, in the context of GNN-based collaborative filtering, such a fixed mapping omits the semantic correlations between entities that are evident in the user-item interaction graph, leading to suboptimal recommendation performance. To this end, we propose Lightweight Embeddings for Graph Collaborative Filtering (LEGCF), a parameter-efficient embedding framework dedicated to GNN-based recommenders. LEGCF innovatively introduces an assignment matrix as an extra learnable component on top of meta-embeddings. To jointly optimize these two heavily entangled components, aside from learning the meta-embeddings by minimizing the recommendation loss, LEGCF further performs efficient assignment update by enforcing a novel semantic similarity constraint and finding its closed-form solution based on matrix pseudo-inverse. The meta-embeddings and assignment matrix are alternately updated, where the latter is sparsified on the fly to ensure negligible storage overhead. Extensive experiments on three benchmark datasets have verified LEGCF's smallest trade-off between size and performance, with consistent accuracy gain over state-of-the-art baselines. The codebase of LEGCF is available in https://github.com/xurong-liang/LEGCF.

Figures (4)

• Figure 1: Key components of LEGCF. Corresponding details can be found in Section \ref{['sec:comp_emb']} for (a), Section \ref{['sec:meta_emb_learning']} for (b) and (c), and Section \ref{['sec:weight_update']} for (d). Note that we use $\textbf{e}$ and $\textbf{s}$ to denote one row of the codebook $\textbf{E}_{meta}$ and assignment matrix $\textbf{S}$, respectively.
• Figure 2: Visualization of sampled entity embeddings and their anchor meta-embeddings. The N@20 value annotated at the top of each plot is the NDCG@20 metric evaluated under the nominated setting. The sampled entities are represented as dots and each cross is a meta-embedding. The same color is used to scatter sampled entities assigned the identical anchor embedding on initialization. Note that we normalize the embeddings in $[-1, 1]$ scale for visualization consistency.
• Figure 3: Average assignment weights of sampled meta-embeddings. The weights in the graphs have a small magnitude because we are taking the average weights over all users/items in the dataset.
• Figure 4: The performance of LEGCF on w.r.t. various hyperparameter settings.