Semi-supervised Vertex Hunting, with Applications in Network and Text Analysis

TL;DR

The paper tackles semi-supervised vertex hunting (SSVH), where partial soft-label information related to barycentric coordinates is available for a subset of data. It develops an optimization-free estimator for the distortion vector $b$ by constructing an orthogonal-projection based matrix $M(\alpha)$ so that $M(\alpha)b=0$, enabling recovery of barycentric coordinates for labeled points and subsequent regression-based estimation of the simplex vertices. Theoretical results establish sub-Gaussian-noise error bounds with a fast $N^{-1/2}$ convergence rate, improving upon unsupervised VH algorithms, and weaker identifiability requirements. The authors instantiate SSVH in semi-supervised network mixed membership estimation (DCMM) and semi-supervised topic modeling (pLSI), providing scalable algorithms with exact recovery in noiseless regimes and robust performance under noise. Empirical studies on networks and text demonstrate substantial gains over unsupervised VH methods with modest labeling, along with favorable computational efficiency, highlighting the practical impact for large-scale semi-supervised learning tasks.

Abstract

Vertex hunting (VH) is the task of estimating a simplex from noisy data points and has many applications in areas such as network and text analysis. We introduce a new variant, semi-supervised vertex hunting (SSVH), in which partial information is available in the form of barycentric coordinates for some data points, known only up to an unknown transformation. To address this problem, we develop a method that leverages properties of orthogonal projection matrices, drawing on novel insights from linear algebra. We establish theoretical error bounds for our method and demonstrate that it achieves a faster convergence rate than existing unsupervised VH algorithms. Finally, we apply SSVH to two practical settings, semi-supervised network mixed membership estimation and semi-supervised topic modeling, resulting in efficient and scalable algorithms.

Figures (5)

• Figure 1: The identification issue for SP (left) and MVT (right). The grey area is the area covered by $r_1,\ldots,r_n$ (the noiseless point cloud). Left: There exists no $r_i$ on the vertices of the true simplex; consequently, there are multiple simplexes containing the point cloud, and the SP solution is not necessarily the true simplex. Right: the point cloud is a ball, and the MVT solution (the minimum-volume simplex that contains this ball) is not unique and does not include the true simplex.
• Figure 2: The influence of label ratio $N / n$, noise level $\sigma$, and dimension $K$ on error $\|\hat{V} - V\|^2_{\mathcal{F}} / K$. "Balanced Dirichlet" and "Dirichlet w/ Pure Points" correspond to setting (1) and (2) respectively.
• Figure 3: Comparison of the true simplex (orange), SSVH estimate (blue), SP estimate (pink), and SVS estimate (red). The green points are the labeled ones.
• Figure 4: A comparison of VH and VI. The network is simulated from a DCMM with $(n,K)=(3000, 3)$, where 2% of nodes are labeled. Each grey point is a node embedding $\hat{r}_i\in\mathbb{R}^2$. The VH approach relies on existence of pure nodes (which is not satisfied in this example). The VI approach bypasses the pure node assumption (by leveraging on labeled nodes) and yields much better accuracy.
• Figure 5: Why MVT does not work for the example in Figure \ref{['fig:compare']}. In this example, $r_i$'s are contained in a disk inside the true simplex. There are infinitely many MVT solutions (by rotation), but none of them is the true simplex.

Theorems & Definitions (15)

• Theorem 2.1: Main discovery
• Theorem 2.2: Uniqueness
• Lemma 3.1: Non-stochastic bound
• Lemma 3.2: Noise terms
• Theorem 3.1
• Lemma 3.3: Ideal estimator
• Definition 4.1: DCMM model
• Definition 4.2: Semi-supervised MME
• Theorem 4.1: Validity of the algorithm
• Definition 4.3: pLSI model