Probabilistically Aligned View-unaligned Clustering with Adaptive Template Selection

TL;DR

This work tackles the view-unaligned clustering (VuP) problem where cross-view correspondences may be missing or incomplete, characterized by an alignment ratio $\rho\in[0,1]$. It proposes PAVuC-ATS, a framework that integrates learning cross-view anchors and view-specific graphs via a bipartite graph with a probabilistic alignment mechanism implemented as a 2-step Markov-chain transition, together with adaptive template selection. An alternating optimization algorithm updates latent projections $\bm{Q}_i$, anchors $\bm{A}$, graphs $\bm{G}_i$, permutations $\bm{\Pi}_i$, and view weights $\phi_i$; convergence to a local minimum is established. Experiments on six benchmark datasets show that PAVuC-ATS consistently outperforms twelve baselines in ACC and NMI, demonstrating robustness to fully unaligned data and scalability.

Abstract

In most existing multi-view modeling scenarios, cross-view correspondence (CVC) between instances of the same target from different views, like paired image-text data, is a crucial prerequisite for effortlessly deriving a consistent representation. Nevertheless, this premise is frequently compromised in certain applications, where each view is organized and transmitted independently, resulting in the view-unaligned problem (VuP). Restoring CVC of unaligned multi-view data is a challenging and highly demanding task that has received limited attention from the research community. To tackle this practical challenge, we propose to integrate the permutation derivation procedure into the bipartite graph paradigm for view-unaligned clustering, termed Probabilistically Aligned View-unaligned Clustering with Adaptive Template Selection (PAVuC-ATS). Specifically, we learn consistent anchors and view-specific graphs by the bipartite graph, and derive permutations applied to the unaligned graphs by reformulating the alignment between two latent representations as a 2-step transition of a Markov chain with adaptive template selection, thereby achieving the probabilistic alignment. The convergence of the resultant optimization problem is validated both experimentally and theoretically. Extensive experiments on six benchmark datasets demonstrate the superiority of the proposed PAVuC-ATS over the baseline methods.

Figures (6)

• Figure 1: The view-unaligned problem with an alignment ratio of $\rho\in[0,1]$, where digits and letters within circles and regular octagons represent the indices of instances, solid/dashed lines indicate known/unknown cross-view correspondences.
• Figure 2: The PAVuC-ATS framework, as depicted in (a), initially learns anchors and graphs. Subsequently, it aligns $\bm{G}_{2}$ and $\bm{G}_{1}$ through the permutation $\bm{\Pi}_{2}$. To determine the permutation, the alignment between the latent representations $\bm{Q}_{2}\bm{x}_{2}^{j}$ and $\bm{Q}_{1}\bm{x}_{1}^{s}$ is reformulated as a Markov chain that transitions from state $\bm{Q}_{2}\bm{x}_{2}^{j}$ to state $\bm{Q}_{1}\bm{x}_{1}^{s}$ as illustrated in (b), achieving the probabilistic alignment, where $\bm{e}^{j}$ is the $j$-th orthogonal base vector.
• Figure 3: Visualizations of (a) $\bm{X}_{1}$, (b) $\bm{X}_{2}$, (c) $\bm{X}_{3}$, and (d) $\bar{\bm{G}}$ on a subset of the CIFAR-10 dataset with an alignment ratio of $\rho=0$.
• Figure 4: The evaluation metrics (ACC and NMI) versus the alignment ratio on the fully unaligned multi-view datasets: (a) ProteinFold, (b) Wiki, and (c) Caltech-101.
• Figure 5: The evaluation metrics (ACC and NMI) versus the parameters $\alpha$ and $\mu$ on the fully unaligned multi-view datasets: (a) ProteinFold, (b) Caltech101-20, (c) Wiki, (d) Caltech-101, (e) Reuters, and (f) CIFAR-10.

Theorems & Definitions (1)

• Theorem 1: huang2013spectral