Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Deep Modularity Networks with Diversity-Preserving Regularization

Yasmin Salehi, Dennis Giannacopoulos

TL;DR

This work tackles the lack of feature-space diversity in Deep Modularity Networks (DMoN) for graph clustering. It introduces Deep Modularity Networks with Diversity-Preserving Regularization (DMoN-DPR), adding distance-based, variance-based, and entropy-based penalties to promote inter-cluster separation and assignment dispersion. On feature-rich benchmarks such as Coauthor CS and Coauthor Physics, DMoN-DPR substantially improves label-aligned metrics like NMI and F1 while maintaining competitive conductance and modularity; statistical tests (p ≤ 0.05) support the gains on these datasets. The results indicate that dataset characteristics, particularly feature richness, govern the effectiveness of diversity regularization and point to future work of adaptive weight learning to automate tuning across varied graphs.

Abstract

Graph clustering plays a crucial role in graph representation learning but often faces challenges in achieving feature-space diversity. While Deep Modularity Networks (DMoN) leverage modularity maximization and collapse regularization to ensure structural separation, they lack explicit mechanisms for feature-space separation, assignment dispersion, and assignment-confidence control. We address this limitation by proposing Deep Modularity Networks with Diversity-Preserving Regularization (DMoN-DPR), which introduces three novel regularization terms: distance-based for inter-cluster separation, variance-based for per-cluster assignment dispersion, and an assignment-entropy penalty with a small positive weight, encouraging more confident assignments gradually. Our method significantly enhances label-based clustering metrics on feature-rich benchmark datasets (paired two-tailed t-test, $p\leq0.05$), demonstrating the effectiveness of incorporating diversity-preserving regularizations in creating meaningful and interpretable clusters.

Deep Modularity Networks with Diversity-Preserving Regularization

TL;DR

This work tackles the lack of feature-space diversity in Deep Modularity Networks (DMoN) for graph clustering. It introduces Deep Modularity Networks with Diversity-Preserving Regularization (DMoN-DPR), adding distance-based, variance-based, and entropy-based penalties to promote inter-cluster separation and assignment dispersion. On feature-rich benchmarks such as Coauthor CS and Coauthor Physics, DMoN-DPR substantially improves label-aligned metrics like NMI and F1 while maintaining competitive conductance and modularity; statistical tests (p ≤ 0.05) support the gains on these datasets. The results indicate that dataset characteristics, particularly feature richness, govern the effectiveness of diversity regularization and point to future work of adaptive weight learning to automate tuning across varied graphs.

Abstract

Graph clustering plays a crucial role in graph representation learning but often faces challenges in achieving feature-space diversity. While Deep Modularity Networks (DMoN) leverage modularity maximization and collapse regularization to ensure structural separation, they lack explicit mechanisms for feature-space separation, assignment dispersion, and assignment-confidence control. We address this limitation by proposing Deep Modularity Networks with Diversity-Preserving Regularization (DMoN-DPR), which introduces three novel regularization terms: distance-based for inter-cluster separation, variance-based for per-cluster assignment dispersion, and an assignment-entropy penalty with a small positive weight, encouraging more confident assignments gradually. Our method significantly enhances label-based clustering metrics on feature-rich benchmark datasets (paired two-tailed t-test, ), demonstrating the effectiveness of incorporating diversity-preserving regularizations in creating meaningful and interpretable clusters.
Paper Structure (40 sections, 7 equations, 4 figures, 56 tables)

This paper contains 40 sections, 7 equations, 4 figures, 56 tables.

Table of Contents

  1. Introduction
  2. Why diversity matters.
  3. Present Work
  4. Related Work
  5. Deep Modularity Networks with Diversity-Preserving Regularization
  6. Distance-Based Regularization
  7. Variance-Based Regularization
  8. Entropy-Based Regularization
  9. Experiments
  10. Evaluation Protocol
  11. Baseline & Implementation Details
  12. Results
  13. Effectiveness of DMoN-DPR in Clustering
  14. Cora, CiteSeer, and PubMed.
  15. Coauthor CS and Coauthor Physics.
  16. ...and 25 more sections

Figures (4)

  • Figure 1: Visualization of clusters (circles) vs. ground truth labels (crosses) formed by DMoN on (a) Cora, (b) CiteSeer, (c) PubMed, (d) Coauthor CS, and (e) Coauthor Physics, and by DMoN--DPR on (f) Cora, (g) CiteSeer, (h) PubMed, (i) Coauthor CS, and (j) Coauthor Physics datasets.
  • Figure 2: Finding the best value of epsilon ($\epsilon$) by setting $W_{\text{dist}}$ to 1 and $W_{\text{var}}$ and $W_{\text{entropy}}$ to zero on the (a) Cora, (b) CiteSeer, (c) PubMed, (d) Coauthor CS, and (e) Coauthor Physics datasets.
  • Figure 3: Finding the best value of $W_{\text{var}}$ by setting $W_{\text{dist}}$ and $W_{\text{entropy}}$ to zero on the (a) Cora, (b) CiteSeer, (c) PubMed, (d) Coauthor CS, and (e) Coauthor Physics datasets.
  • Figure 4: Finding the best value of $W_{\text{entropy}}$ by setting $W_{\text{dist}}$ and $W_{\text{var}}$ to zero on the (a) Cora, (b) CiteSeer, (c) PubMed, (d) Coauthor CS, and (e) Coauthor Physics datasets.