Towards Robust Nonlinear Subspace Clustering: A Kernel Learning Approach

TL;DR

DKLM tackles nonlinear subspace clustering by learning a data-driven kernel $\boldsymbol{\mathcal{K}}$ from self-representation $\mathbf{Z}$ in a RKHS to preserve local manifolds and promote a block-diagonal affinity. It integrates a block-diagonal regularizer $\|\mathbf{Z}\|_{\boxed{k}}$ and a negative trace term $-\mathrm{Tr}(\boldsymbol{\mathcal{K}}\mathbf{Z})$, and solves via alternating updates with a Nyström-based kernel approximation for scalability. The framework establishes connections to kernel k-means and low-rank/self-representation methods through a multiplicative triangle inequality and permutation invariance properties. Empirical results on synthetic, image, text, motion, and time-series data show superior robustness and clustering accuracy compared to state-of-the-art approaches, highlighting the practical impact for real-world nonlinear data analysis.

Abstract

Kernel-based subspace clustering, which addresses the nonlinear structures in data, is an evolving area of research. Despite noteworthy progressions, prevailing methodologies predominantly grapple with limitations relating to (i) the influence of predefined kernels on model performance; (ii) the difficulty of preserving the original manifold structures in the nonlinear space; (iii) the dependency of spectral-type strategies on the ideal block diagonal structure of the affinity matrix. This paper presents DKLM, a novel paradigm for kernel-induced nonlinear subspace clustering. DKLM provides a data-driven approach that directly learns the kernel from the data's self-representation, ensuring adaptive weighting and satisfying the multiplicative triangle inequality constraint, which enhances the robustness of the learned kernel. By leveraging this learned kernel, DKLM preserves the local manifold structure of data in a nonlinear space while promoting the formation of an optimal block-diagonal affinity matrix. A thorough theoretical examination of DKLM reveals its relationship with existing clustering paradigms. Comprehensive experiments on synthetic and real-world datasets demonstrate the effectiveness of the proposed method.

Figures (8)

• Figure 1: To illustrate Corollary 1, consider data points ($\mathbf{x}_1,\cdots, \mathbf{x}_{10}$), organized into two distinct clusters. Since $\mathbf{W}$ is symmetric, we only need to examine one triangle (upper or lower) for this representation. In the topological view, integers 1–10 correspond to $\mathbf{x}_1\ -\ \mathbf{x}_{10}$, respectively, with edge weights derived from $\mathbf{W}$. Observing the example, nodes 3 and 7, with higher degrees, result in $\mathbf{G}_{37} =0.2/\sqrt{1.6*1.8}\thickapprox 0.1179$, indicating reduced similarity between clusters. Conversely, similarity within the same cluster is enhanced, as shown by $\mathbf{G}_{12}=0.6/\sqrt{0.6*1.2}\thickapprox 0.7071$.
• Figure 2: Visualization of results on three 2D datasets: (a), (c), and (e) represent the three synthetic datasets, while (b), (d), and (f) show the affinity graphs $\mathbf{Z}$ learned by the model. For clarity, only edges with weights of 0.001 or higher are displayed.
• Figure 3: Binarized affinity matrices on high-dimensional SyD4, comparing various SOTA methods with ours.
• Figure 4: Sample images from Yale, ORL, Jaffe, COIL20, Binary Alphadigits, and Hopkins-155 datasets
• Figure 5: Clustering results for four sequences (people1, cars10, 1R2TCR, 2T3RTCR) from the Hopkins155 database. The top row shows sample images with overlaid tracked points, the second row displays self-representation matrices generated by DKLM, and the bottom row provides 3D visualizations of the clustering outcomes.

Theorems & Definitions (20)

• Theorem 1: von2007tutorial, Proposition 4
• Corollary 1: Degree-scaled weighting
• Theorem 2
• Proof 1
• Theorem 3
• Proof 2
• Theorem 4
• Theorem 5
• Proof 3
• Theorem 6