To Share or Not to Share: Investigating Weight Sharing in Variational Graph Autoencoders

TL;DR

The paper addresses whether weight sharing (WS) across hidden layers in variational graph autoencoders (VGAE) is beneficial. Through an extensive empirical study of 10 VGAE variants across 16 diverse graphs on link prediction and community detection tasks, it shows that WS consistently matches or closely approaches the performance of non-WS models while reducing parameters and training time. The findings argue that WS provides parameter efficiency and regularization without sacrificing accuracy, supporting its adoption in VGAE designs. The authors also release their code publicly to facilitate reproducibility and further exploration of WS in graph representation learning.

Abstract

This paper investigates the understudied practice of weight sharing (WS) in variational graph autoencoders (VGAE). WS presents both benefits and drawbacks for VGAE model design and node embedding learning, leaving its overall relevance unclear and the question of whether it should be adopted unresolved. We rigorously analyze its implications and, through extensive experiments on a wide range of graphs and VGAE variants, demonstrate that the benefits of WS consistently outweigh its drawbacks. Based on our findings, we recommend WS as an effective approach to optimize, regularize, and simplify VGAE models without significant performance loss.