OCN: Effectively Utilizing Higher-Order Common Neighbors for Better Link Prediction

TL;DR

The paper tackles the challenge of leveraging higher-order common neighbors for link prediction by identifying redundancy across CN orders and over-smoothing in their representations. It introduces Orthogonal Common Neighbor (OCN), built via coefficient orthogonalization and path-based normalization, and an efficient variant OCNP based on polynomial filters. The authors provide theoretical analyses and extensive ablations, showing that orthogonalization reduces redundancy while normalization mitigates over-smoothing, enabling higher-order CNs to meaningfully contribute to prediction. Empirically, OCN/OCNP achieve state-of-the-art results on seven real-world datasets, including large-scale Open Graph Benchmark graphs, with notable improvements over baselines and favorable scalability, especially for OCNP.

Abstract

Common Neighbors (CNs) and their higher-order variants are important pairwise features widely used in state-of-the-art link prediction methods. However, existing methods often struggle with the repetition across different orders of CNs and fail to fully leverage their potential. We identify that these limitations stem from two key issues: redundancy and over-smoothing in high-order common neighbors. To address these challenges, we design orthogonalization to eliminate redundancy between different-order CNs and normalization to mitigate over-smoothing. By combining these two techniques, we propose Orthogonal Common Neighbor (OCN), a novel approach that significantly outperforms the strongest baselines by an average of 7.7% on popular link prediction benchmarks. A thorough theoretical analysis is provided to support our method. Ablation studies also verify the effectiveness of our orthogonalization and normalization techniques.

Figures (10)

• Figure 1: The heatmap illustrates the correlation coefficients between different orders of $k$-order Common Neighbors (CNs), highlighting increasing redundancy with higher orders.
• Figure 2: (a) Using the Cora dataset, we measure the coefficient of variation (CV) of common neighbors (CNs) at different orders. Higher CV indicates less over-smoothing. Results show increasing over-smoothing with higher orders, but our method (yellow) alleviates this issue. (b) The impact of $k$-hop CNs on the upper bound of $d_{ij}$: $k$-hop CNs alone have no effect (blue line). With normalizedCN (yellow line), the bound tightens as each $k$-hop CN contributes $\sum_{c \in CN^k(i, j)} \frac{1}{|P_k(c)|}$ instead of 1.
• Figure 3: White, green, orange, and yellow represent node features 0, 1, 2, and 3, respectively. $v_2$ and $v_3$ are symmetric, and GAE cannot distinguish $(v_1, v_2)$ and $(v_1, v_3)$. With node features ignored, $(v_1, v_2)$ and $(v_1, v_3)$ are symmetric, so CN, RA, AA, Neo-GNN, and BUDDY cannot distinguish them. NCN also degenerates into GAE, so it also cannot. However, $(v_1, v_2)$ and $(v_1, v_3)$ have different 2-hop CNs, which allows OCN to distinguish them.
• Figure 4: Inference time and GPU memory on ogbl-collab. The measured process includes preprocessing and predicting one batch of test links. As shown in \ref{['ApendixI']}, the relation between time $y$ and batch size $t$ is $y = B + Ct$.
• Figure 5: This figure demonstrates that orthogonalization reduces redundancy in higher-order common neighbors, as shown by the increased and concentrated JSD values, making the representations more independent and distinguishable.

Theorems & Definitions (21)

• Definition 5.1
• Theorem 5.2
• Proposition 5.3
• Proposition 5.4
• Theorem 6.1
• proof
• Definition D.1
• Definition D.2
• Lemma D.3
• proof