Multiview learning with twin parametric margin SVM

TL;DR

This work tackles multiview classification under heteroscedastic noise by introducing MvTPMSVM, which constructs four parametric-margin hyperplanes across two views and directly leverages kernel methods without large matrix inversions. The model regularizes each view’s margin hyperplanes, aligns classifiers via an $ ext{epsilon}$-insensitive constraint, and leverages two small quadratic programs to achieve substantial computational efficiency relative to existing MVL methods. Extensive experiments on synthetic data, 55 UCI/KEEL datasets, and 45 AwA binary-class datasets demonstrate superior generalization performance and favorable statistical rankings, with robustness to hyperparameter settings except in extreme regimes. The approach advances multiview learning by combining marginals from two views with heteroscedastic-noise awareness, enabling scalable, accurate, and kernelizable multiview classification; code is publicly available at the provided repository.

Abstract

Multiview learning (MVL) seeks to leverage the benefits of diverse perspectives to complement each other, effectively extracting and utilizing the latent information within the dataset. Several twin support vector machine-based MVL (MvTSVM) models have been introduced and demonstrated outstanding performance in various learning tasks. However, MvTSVM-based models face significant challenges in the form of computational complexity due to four matrix inversions, the need to reformulate optimization problems in order to employ kernel-generated surfaces for handling non-linear cases, and the constraint of uniform noise assumption in the training data. Particularly in cases where the data possesses a heteroscedastic error structure, these challenges become even more pronounced. In view of the aforementioned challenges, we propose multiview twin parametric margin support vector machine (MvTPMSVM). MvTPMSVM constructs parametric margin hyperplanes corresponding to both classes, aiming to regulate and manage the impact of the heteroscedastic noise structure existing within the data. The proposed MvTPMSVM model avoids the explicit computation of matrix inversions in the dual formulation, leading to enhanced computational efficiency. We perform an extensive assessment of the MvTPMSVM model using benchmark datasets such as UCI, KEEL, synthetic, and Animals with Attributes (AwA). Our experimental results, coupled with rigorous statistical analyses, confirm the superior generalization capabilities of the proposed MvTPMSVM model compared to the baseline models. The source code of the proposed MvTPMSVM model is available at \url{https://github.com/mtanveer1/MvTPMSVM}.

Figures (7)

• Figure 1: Flow diagram of the proposed MvTPMSVM model.
• Figure 2: Illustration of three synthetic multiview datasets.
• Figure 3: The performance comparison of the proposed MvTPMSVM model across various intervals of the training samples using the AwA datasets.
• Figure 4: Performance comparison of proposed MvTPMSVM model along with the baseline models on UCI and KEEL datasets using Seny, Spey, and Pren
• Figure 5: Performance comparison of the proposed MvTPMSVM model along with the baseline models on AwA datasets using Seny, Spey, and Pren