Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Multi-Task Learning Based on Support Vector Machines and Twin Support Vector Machines: A Comprehensive Survey

Fatemeh Bazikar, Hossein Moosaei, Atefeh Hemmati, Panos M. Pardalos

TL;DR

This survey synthesizes margin-based multi-task learning approaches built on SVMs and TWSVMs, highlighting how shared representations and task regularization enable cross-task knowledge transfer while maintaining interpretability. It contrasts standard SVM-based MTL (M-SVM and extensions) with DMTSVM-based MTL, emphasizing trade-offs between generalization and computation. The work catalogs formulations, dualities, and kernelizations, surveys a broad spectrum of extensions (e.g., universum data, robust losses, safe screening), and discusses applications across vision, NLP, and bioinformatics. It also outlines clear directions for scalability, deeper integration with deep models, robust multi-class/structured outputs, and explainability to guide future margin-based MTL research and practice.

Abstract

Multi-task learning (MTL) enables simultaneous training across related tasks, leveraging shared information to improve generalization, efficiency, and robustness, especially in data-scarce or high-dimensional scenarios. While deep learning dominates recent MTL research, Support Vector Machines (SVMs) and Twin SVMs (TWSVMs) remain relevant due to their interpretability, theoretical rigor, and effectiveness with small datasets. This chapter surveys MTL approaches based on SVM and TWSVM, highlighting shared representations, task regularization, and structural coupling strategies. Special attention is given to emerging TWSVM extensions for multi-task settings, which show promise but remain underexplored. We compare these models in terms of theoretical properties, optimization strategies, and empirical performance, and discuss applications in fields such as computer vision, natural language processing, and bioinformatics. Finally, we identify research gaps and outline future directions for building scalable, interpretable, and reliable margin-based MTL frameworks. This work provides a comprehensive resource for researchers and practitioners interested in SVM- and TWSVM-based multi-task learning.

Multi-Task Learning Based on Support Vector Machines and Twin Support Vector Machines: A Comprehensive Survey

TL;DR

This survey synthesizes margin-based multi-task learning approaches built on SVMs and TWSVMs, highlighting how shared representations and task regularization enable cross-task knowledge transfer while maintaining interpretability. It contrasts standard SVM-based MTL (M-SVM and extensions) with DMTSVM-based MTL, emphasizing trade-offs between generalization and computation. The work catalogs formulations, dualities, and kernelizations, surveys a broad spectrum of extensions (e.g., universum data, robust losses, safe screening), and discusses applications across vision, NLP, and bioinformatics. It also outlines clear directions for scalability, deeper integration with deep models, robust multi-class/structured outputs, and explainability to guide future margin-based MTL research and practice.

Abstract

Multi-task learning (MTL) enables simultaneous training across related tasks, leveraging shared information to improve generalization, efficiency, and robustness, especially in data-scarce or high-dimensional scenarios. While deep learning dominates recent MTL research, Support Vector Machines (SVMs) and Twin SVMs (TWSVMs) remain relevant due to their interpretability, theoretical rigor, and effectiveness with small datasets. This chapter surveys MTL approaches based on SVM and TWSVM, highlighting shared representations, task regularization, and structural coupling strategies. Special attention is given to emerging TWSVM extensions for multi-task settings, which show promise but remain underexplored. We compare these models in terms of theoretical properties, optimization strategies, and empirical performance, and discuss applications in fields such as computer vision, natural language processing, and bioinformatics. Finally, we identify research gaps and outline future directions for building scalable, interpretable, and reliable margin-based MTL frameworks. This work provides a comprehensive resource for researchers and practitioners interested in SVM- and TWSVM-based multi-task learning.

Paper Structure

This paper contains 26 sections, 50 equations, 4 figures, 4 tables.

Table of Contents

  1. Fundamentals of Multi-Task Learning
  2. Definitions and Learning Settings
  3. Categories of Multi-Task Learning Approaches
  4. Advantages and Challenges
  5. Support Vector Machines: A Primer
  6. Linear SVM Formulation
  7. Nonlinear Extensions
  8. Twin Support Vector Machines: A Primer
  9. Linear TWSVM Formulation
  10. Nonlinear Extensions
  11. Comparison between Standard SVM and TWSVM
  12. Multi-Task Learning Based on SVM
  13. Formulation of Multi-Task Learning Based on SVM
  14. Step 1: Derivatives w.r.t. primal variables
  15. Step 2: Dual problem
  16. ...and 11 more sections

Figures (4)

  • Figure 1: Geometric illustration of the SVM framework.
  • Figure 2: Geometric illustration of the TWSVM framework.
  • Figure 3: Geometric illustration of the M-SVM framework.
  • Figure 4: Geometric illustration of the DMTSVM framework.

Theorems & Definitions (3)

  • remark 1
  • remark 2
  • remark 3