Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Mixed membership distribution-free model

Huan Qing, Jingli Wang

TL;DR

We address community detection in overlapping weighted networks by proposing the Mixed Membership Distribution-Free (MMDF) model, which allows arbitrary real-valued edge weights with mean structure $\Omega = \rho \Pi P \Pi^T$. A spectral DFSP algorithm (Ideal DFSP and DFSP) recovers node memberships under MMDF with provable consistency, and a fuzzy weighted modularity $Q_{\mathcal{M}}(k)$ enables estimation of the number of communities. MMDF subsumes classical models like MMSB and SBM and extends to signed and sparse networks; the framework remains distribution-free with guarantees under mild conditions. Extensive simulations across Normal, Bernoulli, Poisson, Uniform, and Signed settings, plus nine real-world networks, demonstrate accuracy, robustness, and computational efficiency, showing advantages over several baselines in detection and model selection.

Abstract

We consider the problem of community detection in overlapping weighted networks, where nodes can belong to multiple communities and edge weights can be finite real numbers. To model such complex networks, we propose a general framework - the mixed membership distribution-free (MMDF) model. MMDF has no distribution constraints of edge weights and can be viewed as generalizations of some previous models, including the well-known mixed membership stochastic blockmodels. Especially, overlapping signed networks with latent community structures can also be generated from our model. We use an efficient spectral algorithm with a theoretical guarantee of convergence rate to estimate community memberships under the model. We also propose the fuzzy weighted modularity to evaluate the quality of community detection for overlapping weighted networks with positive and negative edge weights. We then provide a method to determine the number of communities for weighted networks by taking advantage of our fuzzy weighted modularity. Numerical simulations and real data applications are carried out to demonstrate the usefulness of our mixed membership distribution-free model and our fuzzy weighted modularity.

Mixed membership distribution-free model

TL;DR

We address community detection in overlapping weighted networks by proposing the Mixed Membership Distribution-Free (MMDF) model, which allows arbitrary real-valued edge weights with mean structure . A spectral DFSP algorithm (Ideal DFSP and DFSP) recovers node memberships under MMDF with provable consistency, and a fuzzy weighted modularity enables estimation of the number of communities. MMDF subsumes classical models like MMSB and SBM and extends to signed and sparse networks; the framework remains distribution-free with guarantees under mild conditions. Extensive simulations across Normal, Bernoulli, Poisson, Uniform, and Signed settings, plus nine real-world networks, demonstrate accuracy, robustness, and computational efficiency, showing advantages over several baselines in detection and model selection.

Abstract

We consider the problem of community detection in overlapping weighted networks, where nodes can belong to multiple communities and edge weights can be finite real numbers. To model such complex networks, we propose a general framework - the mixed membership distribution-free (MMDF) model. MMDF has no distribution constraints of edge weights and can be viewed as generalizations of some previous models, including the well-known mixed membership stochastic blockmodels. Especially, overlapping signed networks with latent community structures can also be generated from our model. We use an efficient spectral algorithm with a theoretical guarantee of convergence rate to estimate community memberships under the model. We also propose the fuzzy weighted modularity to evaluate the quality of community detection for overlapping weighted networks with positive and negative edge weights. We then provide a method to determine the number of communities for weighted networks by taking advantage of our fuzzy weighted modularity. Numerical simulations and real data applications are carried out to demonstrate the usefulness of our mixed membership distribution-free model and our fuzzy weighted modularity.
Paper Structure (25 sections, 6 theorems, 23 equations, 6 figures, 4 tables, 2 algorithms)

This paper contains 25 sections, 6 theorems, 23 equations, 6 figures, 4 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. The Mixed Membership Distribution-Free Model
  3. Algorithm
  4. Asymptotic Consistency
  5. Examples
  6. Missing edge
  7. Estimation of the Number of Communities
  8. Experimental Results
  9. Comparison algorithms
  10. Evaluation metrics
  11. Simulations
  12. Normal distribution
  13. Bernoulli distribution
  14. Poisson distribution
  15. Uniform distribution
  16. ...and 10 more sections

Key Result

Proposition 1

(Identifiability). MMDF is identifiable: For eligible $(P,\Pi)$ and $(\tilde{P}, \tilde{\Pi})$, if $\rho \Pi P\Pi'=\rho \tilde{\Pi}\tilde{P}\tilde{\Pi}'$, then $(\Pi,P)$ and $(\tilde{\Pi},\tilde{P})$ are identical up to a permutation of the $K$ communities.

Figures (6)

  • Figure 1: Numerical results for simulations.
  • Figure 2: Adjacency matrices of Gahuku-Gama subtribes, Karate-club-weighted, and Slovene Parliamentary Party.
  • Figure 3: Fuzzy weighted modularity $Q$ computed by Equation (\ref{['Modularity']}) against the number of clusters by DFSP for real-world networks in Table \ref{['realdata']}.
  • Figure 4: Communities detected by DFSP. Colors indicate communities and the black square indicates highly mixed nodes, where communities are obtained by $\hat{c}$. For visualization, we do not show edge weights and node labels for US Top-500 Airport Network, Political blogs, and US airports.
  • Figure 5: Numerical results of changing $\delta$.
  • ...and 1 more figures

Theorems & Definitions (23)

  • Definition 1
  • Proposition 1
  • Remark 1
  • Lemma 1
  • Remark 2
  • Theorem 1
  • Corollary 1
  • Example 1
  • Example 2
  • Example 3
  • ...and 13 more