DGR: A General Graph Desmoothing Framework for Recommendation via Global and Local Perspectives

TL;DR

This work tackles the persistent over-smoothing problem in deep GCN-based recommender systems, which blurs user and item embeddings and harms personalization. It presents DGR, a model-free desmoothing framework with two components: GMP, which perturbs embeddings at each layer to move away from the global over-smoothing point, and LEC, which preserves local readout similarities by leveraging similar and marginal neighbors. The training objective combines the standard CF loss with a dedicated LEC term, and the approach is demonstrated to generalize across five backbone models and five public datasets, achieving consistent improvements and better stability in deeper architectures. The work offers a practical, architecture-agnostic solution with clear potential impact for deploying more personalized recommendations in real systems.

Abstract

Graph Convolutional Networks (GCNs) have become pivotal in recommendation systems for learning user and item embeddings by leveraging the user-item interaction graph's node information and topology. However, these models often face the famous over-smoothing issue, leading to indistinct user and item embeddings and reduced personalization. Traditional desmoothing methods in GCN-based systems are model-specific, lacking a universal solution. This paper introduces a novel, model-agnostic approach named \textbf{D}esmoothing Framework for \textbf{G}CN-based \textbf{R}ecommendation Systems (\textbf{DGR}). It effectively addresses over-smoothing on general GCN-based recommendation models by considering both global and local perspectives. Specifically, we first introduce vector perturbations during each message passing layer to penalize the tendency of node embeddings approximating overly to be similar with the guidance of the global topological structure. Meanwhile, we further develop a tailored-design loss term for the readout embeddings to preserve the local collaborative relations between users and their neighboring items. In particular, items that exhibit a high correlation with neighboring items are also incorporated to enhance the local topological information. To validate our approach, we conduct extensive experiments on 5 benchmark datasets based on 5 well-known GCN-based recommendation models, demonstrating the effectiveness and generalization of our proposed framework.

Figures (13)

• Figure 1: Empirical analysis on the evolution of the node embeddings as the layer number of message-passing increases. The left part shows the $\mathbf{D}(\mathbf{E}^{(k)},\mathbf{M})$ changes across the layer number $k$ in the range [1, 20]. The right part shows the t-SNE visualization of the over-smoothing point $\mathbf{m}$ and the trajectory of embeddings of an item and a user.
• Figure 2: Illustration of the embedding vector updating in Global Desmoothing Message Passing. The solid black line from $\mathbf{e}^{(0)}$ to $\mathbf{m}$ represents the original trajectory of the GCN.
• Figure 3: An illustration of DGR architecture. In the Global Desmoothing Message Passing component, user and item embeddings are put away from the over-smoothing point to preserve node embedding distinctiveness. In the Local Node Embedding Correction component, similar neighbor nodes like node $i_4$ are aggregated together while the marginal nodes like node $i_5$ is segregated in the local graph to utilize the collaborative signals. In the readout phase, $\mathbf{e}^{(0)}$ are also utilized, although they are not shown in the diagram for clarity.
• Figure 4: Training losses with epochs on Douban-Book
• Figure 5: Performance of LightGCN with and without the DGR across a range of model depths. The orange line and blue line represent Recall@20 and NDCG@20, respectively. The solid line and dashed line represent models with and without DGR, respectively.

Theorems & Definitions (1)

• Proposition 1