Generalizable and Robust Spectral Method for Multi-view Representation Learning

TL;DR

SpecRaGE addresses the generalization and scalability limitations of graph Laplacian–based multi-view representation learning by learning a parametric map that jointly diagonalizes multi-view Laplacians without explicit alignment. It introduces a robust fusion module with sample-specific view weighting and self-supervised affinity learning to down-weight noisy views, yielding representations that generalize to new data and scale to large datasets. Theoretical analysis links the loss to approximate joint diagonalization, and experiments across five benchmarks show state-of-the-art clustering and classification performance, with strong robustness to contamination. The framework offers a practical, scalable approach to reliable multi-view learning in real-world noisy settings.

Abstract

Multi-view representation learning (MvRL) has garnered substantial attention in recent years, driven by the increasing demand for applications that can effectively process and analyze data from multiple sources. In this context, graph Laplacian-based MvRL methods have demonstrated remarkable success in representing multi-view data. However, these methods often struggle with generalization to new data and face challenges with scalability. Moreover, in many practical scenarios, multi-view data is contaminated by noise or outliers. In such cases, modern deep-learning-based MvRL approaches that rely on alignment or contrastive objectives present degraded performance in downstream tasks, as they may impose incorrect consistency between clear and corrupted data sources. We introduce $\textit{SpecRaGE}$, a novel fusion-based framework that integrates the strengths of graph Laplacian methods with the power of deep learning to overcome these challenges. SpecRage uses neural networks to learn parametric mapping that approximates a joint diagonalization of graph Laplacians. This solution bypasses the need for alignment while enabling generalizable and scalable learning of informative and meaningful representations. Moreover, it incorporates a meta-learning fusion module that dynamically adapts to data quality, ensuring robustness against outliers and noisy views. Our extensive experiments demonstrate that SpecRaGE outperforms state-of-the-art methods, particularly in scenarios with data contamination, paving the way for more reliable and efficient multi-view learning.

Figures (8)

• Figure 1: SpecRaGE architecture: Given an input of $V$ batches, each containing $m$ samples, $V$ view-specific representations are generated by $V$ corresponding neural networks. Subsequently, the concatenated input is passed through the fusion model, which computes the weights for performing the weighted sum fusion. The resulting fused representation undergoes QR decomposition to enforce orthogonality. Finally, the loss function in Eq. \ref{['loss:4']} is computed, and the weights of all networks are updated via the gradients.
• Figure 2: Distribution of weights assigned by our weighting model on the scikit-learn 2D Blobs dataset. Clean samples (blue) receive moderate weights centered around 0.5, while outliers (red), are assigned very low weights, effectively reducing their influence on the learning process.
• Figure 3: Visualization of the unified representation $Y$ during training on the BDGP dataset.
• Figure 4: Clustering performance (ACC) degradation across various datasets under increasing levels of outliers (top) and noise (bottom). Each curve represents a different method, illustrating the relative performance drop (%) as the proportion of outliers or noise rises
• Figure 5: Comparison of clustering performance (ACC) degradation among different fusion methods on the BDGP dataset with increasing outlier and noise ratios.

Theorems & Definitions (5)

• Definition 1
• Proposition 1
• Corollary 1
• proof
• proof