Estimating Higher-Order Mixed Memberships via the $\ell_{2,\infty}$ Tensor Perturbation Bound

TL;DR

The tensor mixed-membership blockmodel is proposed, a generalization of the tensor blockmodel positing that memberships need not be discrete, but instead are convex combinations of latent communities.

Abstract

Higher-order multiway data is ubiquitous in machine learning and statistics and often exhibits community-like structures, where each component (node) along each different mode has a community membership associated with it. In this paper we propose the tensor mixed-membership blockmodel, a generalization of the tensor blockmodel positing that memberships need not be discrete, but instead are convex combinations of latent communities. We establish the identifiability of our model and propose a computationally efficient estimation procedure based on the higher-order orthogonal iteration algorithm (HOOI) for tensor SVD composed with a simplex corner-finding algorithm. We then demonstrate the consistency of our estimation procedure by providing a per-node error bound, which showcases the effect of higher-order structures on estimation accuracy. To prove our consistency result, we develop the $\ell_{2,\infty}$ tensor perturbation bound for HOOI under independent, heteroskedastic, subgaussian noise that may be of independent interest. Our analysis uses a novel leave-one-out construction for the iterates, and our bounds depend only on spectral properties of the underlying low-rank tensor under nearly optimal signal-to-noise ratio conditions such that tensor SVD is computationally feasible. Finally, we apply our methodology to real and simulated data, demonstrating some effects not identifiable from the model with discrete community memberships.

Figures (7)

• Figure 1: Pure node memberships for the countries, with red corresponding to higher membership intensity. Grey corresponds to countries that were not included in the analysis.
• Figure 2: Simulated maximum node-wise errors, as described in \ref{['sec:sims']}. The left figure depicts relative $\ell_{2,\infty}$ estimation error with varying levels of $\sigma$ averaged over 10 runs, and the right hand figure examines relative $\ell_{2,\infty}$ estimation error averaged across $p$ with varying levels of heteroskedasticity.
• Figure 3: Community memberships for airports (left) and airlines (right), separated according to country to emphasize "disconnectedness" between Chinese airports and airlines with American airports and airlines.
• Figure 4: Pure node memberships for the time mode, with higher values corresponding to stronger membership intensity. Data are smoothed within each year to emphasize the effect of seasonality
• Figure 5: Pure node memberships for the airport mode, with pure nodes ATL (top left), LAX (top right), and LGA (bottom left). Red demonstrates high membership and purple demonstrates low membership within that particular community. The pure nodes are drawn with large triangles.

Theorems & Definitions (60)

• Proposition 1: Identifiability
• Proposition 2
• Remark 1: Other Vertex Hunting Procedures
• Remark 2: Comparison to Prior Works
• Lemma 1
• Remark 3: Relation to Tensor Subspace Estimation
• Theorem 1: Uniform Estimation Error
• Remark 4: Relationship to Matrix Mixed-Membership Blockmodels
• Remark 5: Extension to Bernoulli Noise
• Corollary 1: Average $\ell_1$ Error