Scalable Robust Bayesian Co-Clustering with Compositional ELBOs
Ashwin Vinod, Chandrajit Bajaj
TL;DR
This work addresses robust co-clustering in high-dimensional and noisy data by learning row and column clusters directly in latent space. It introduces a scalable, fully variational framework with two VAEs (one for instances, one for features) equipped with Gaussian Mixture Model priors, a scale-based mechanism to counteract posterior collapse, and a joint cell-space latent variable to capture local interactions. A mutual information-based cross-loss enforces coherent alignment between learned row and column partitions, while a compositional ELBO with doubly reparameterized gradients stabilizes training. Empirical results across image, text, and biomedical-like data show improved ACC and NMI over baselines and demonstrate robustness to corruption and missing data, highlighting the method’s practical applicability to multi-modal clustering tasks. The approach also supports potential biomedical extensions, as illustrated by an appendix applying the model to Parkinson’s disease-related multimodal data (PPMI).
Abstract
Co-clustering exploits the duality of instances and features to simultaneously uncover meaningful groups in both dimensions, often outperforming traditional clustering in high-dimensional or sparse data settings. Although recent deep learning approaches successfully integrate feature learning and cluster assignment, they remain susceptible to noise and can suffer from posterior collapse within standard autoencoders. In this paper, we present the first fully variational Co-clustering framework that directly learns row and column clusters in the latent space, leveraging a doubly reparameterized ELBO to improve gradient signal-to-noise separation. Our unsupervised model integrates a Variational Deep Embedding with a Gaussian Mixture Model (GMM) prior for both instances and features, providing a built-in clustering mechanism that naturally aligns latent modes with row and column clusters. Furthermore, our regularized end-to-end noise learning Compositional ELBO architecture jointly reconstructs the data while regularizing against noise through the KL divergence, thus gracefully handling corrupted or missing inputs in a single training pipeline. To counteract posterior collapse, we introduce a scale modification that increases the encoder's latent means only in the reconstruction pathway, preserving richer latent representations without inflating the KL term. Finally, a mutual information-based cross-loss ensures coherent co-clustering of rows and columns. Empirical results on diverse real-world datasets from multiple modalities, numerical, textual, and image-based, demonstrate that our method not only preserves the advantages of prior Co-clustering approaches but also exceeds them in accuracy and robustness, particularly in high-dimensional or noisy settings.