Enhancing Multiview Synergy: Robust Learning by Exploiting the Wave Loss Function with Consensus and Complementarity Principles

TL;DR

This work addresses the robustness gaps in multiview SVMs by introducing Wave-MvSVM, a framework that jointly leverages the consensus and complementarity principles through a novel wave loss. The $\mathcal{L}_{wave}$ is bounded, smooth, and classification-calibrated, enabling stable learning in the presence of noisy or view-inconsistent data, while a between-view co-regularization term and adaptive view weighting enhance discriminative power. The optimization is efficiently solved via a hybrid GD-ADMM scheme, with theoretical generalization guarantees grounded in Rademacher complexity. Empirical results on UCI/KEEL benchmarks and AwA demonstrate superior accuracy and robustness to label noise, supported by rigorous statistical tests, underscoring Wave-MvSVM’s practical impact for robust multiview classification.

Abstract

Multiview learning (MvL) is an advancing domain in machine learning, leveraging multiple data perspectives to enhance model performance through view-consistency and view-discrepancy. Despite numerous successful multiview-based SVM models, existing frameworks predominantly focus on the consensus principle, often overlooking the complementarity principle. Furthermore, they exhibit limited robustness against noisy, error-prone, and view-inconsistent samples, prevalent in multiview datasets. To tackle the aforementioned limitations, this paper introduces Wave-MvSVM, a novel multiview support vector machine framework leveraging the wave loss (W-loss) function, specifically designed to harness both consensus and complementarity principles. Unlike traditional approaches that often overlook the complementary information among different views, the proposed Wave-MvSVM ensures a more comprehensive and resilient learning process by integrating both principles effectively. The W-loss function, characterized by its smoothness, asymmetry, and bounded nature, is particularly effective in mitigating the adverse effects of noisy and outlier data, thereby enhancing model stability. Theoretically, the W-loss function also exhibits a crucial classification-calibrated property, further boosting its effectiveness. Wave-MvSVM employs a between-view co-regularization term to enforce view consistency and utilizes an adaptive combination weight strategy to maximize the discriminative power of each view. The optimization problem is efficiently solved using a combination of GD and the ADMM, ensuring reliable convergence to optimal solutions. Theoretical analyses, grounded in Rademacher complexity, validate the generalization capabilities of the Wave-MvSVM model. Extensive empirical evaluations across diverse datasets demonstrate the superior performance of Wave-MvSVM in comparison to existing benchmark models.

Figures (9)

• Figure 1: An illustration of the W-loss function, where $\lambda$ is fixed at $1$, and different values of $a$.
• Figure 2: Flowchart of the model construction of Wave-MvSVM
• Figure 3: Geometrical depiction of Wave-MvSVM: This figure demonstrates how the model handles misclassified samples, the influence of slack variables, and the application of penalties for outliers and noisy data. It provides a visual understanding of the asymmetric and bounded nature of the W-loss function, showing its impact on the decision boundary and the overall robustness of the proposed model.
• Figure 4: Graphically represent $\int_{Y} \mathcal{L}_{\text{wave}}(1-y\mathcal{F}(x))d\mathcal{P}(y|x)$ as a function of $\mathcal{F}(x)$, considering various values of $\mathcal{P}(x)$. (a) Illustrate the case where $\mathcal{P}(x) > 1/2$. (b) Depict the case when $\mathcal{P}(x)<1/2$.
• Figure 5: ROC curves of the proposed Wave-MvSVM model along with the baseline models on the UCI and KEEL datasets.

Theorems & Definitions (8)

• Theorem 5.1
• proof
• Definition 1
• Lemma 5.2
• Lemma 5.3
• Lemma 5.4
• Theorem 5.5
• proof