Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Parallel Algorithms for Combined Regularized Support Vector Machines: Application in Music Genre Classification

Rongmei Liang, Zizheng Liu, Xiaofei Wu, Jingwen Tu

TL;DR

This paper tackles scalable learning for combined regularized SVMs (CR-SVMs) when data are distributed across multiple machines. It introduces a unified consensus optimization framework solved by a parallel ADMM with Gaussian back-substitution, ensuring convergence and scalability regardless of the chosen loss or regularization. A new SGL-SVM model is proposed to capture group and within-group sparsity, and it is applied to music genre classification. The authors prove convergence properties and loss/regularizer-agnostic complexity, and demonstrate effectiveness through synthetic and FMA-based experiments, highlighting MFCCs as the most informative feature group.

Abstract

In the era of rapid development of artificial intelligence, its applications span across diverse fields, relying heavily on effective data processing and model optimization. Combined Regularized Support Vector Machines (CR-SVMs) can effectively handle the structural information among data features, but there is a lack of efficient algorithms in distributed-stored big data. To address this issue, we propose a unified optimization framework based on consensus structure. This framework is not only applicable to various loss functions and combined regularization terms but can also be effectively extended to non-convex regularization terms, showing strong scalability. Based on this framework, we develop a distributed parallel alternating direction method of multipliers (ADMM) algorithm to efficiently compute CR-SVMs when data is stored in a distributed manner. To ensure the convergence of the algorithm, we also introduce the Gaussian back-substitution method. Meanwhile, for the integrity of the paper, we introduce a new model, the sparse group lasso support vector machine (SGL-SVM), and apply it to music information retrieval. Theoretical analysis confirms that the computational complexity of the proposed algorithm is not affected by different regularization terms and loss functions, highlighting the universality of the parallel algorithm. Experiments on synthetic and free music archiv datasets demonstrate the reliability, stability, and efficiency of the algorithm.

Parallel Algorithms for Combined Regularized Support Vector Machines: Application in Music Genre Classification

TL;DR

This paper tackles scalable learning for combined regularized SVMs (CR-SVMs) when data are distributed across multiple machines. It introduces a unified consensus optimization framework solved by a parallel ADMM with Gaussian back-substitution, ensuring convergence and scalability regardless of the chosen loss or regularization. A new SGL-SVM model is proposed to capture group and within-group sparsity, and it is applied to music genre classification. The authors prove convergence properties and loss/regularizer-agnostic complexity, and demonstrate effectiveness through synthetic and FMA-based experiments, highlighting MFCCs as the most informative feature group.

Abstract

In the era of rapid development of artificial intelligence, its applications span across diverse fields, relying heavily on effective data processing and model optimization. Combined Regularized Support Vector Machines (CR-SVMs) can effectively handle the structural information among data features, but there is a lack of efficient algorithms in distributed-stored big data. To address this issue, we propose a unified optimization framework based on consensus structure. This framework is not only applicable to various loss functions and combined regularization terms but can also be effectively extended to non-convex regularization terms, showing strong scalability. Based on this framework, we develop a distributed parallel alternating direction method of multipliers (ADMM) algorithm to efficiently compute CR-SVMs when data is stored in a distributed manner. To ensure the convergence of the algorithm, we also introduce the Gaussian back-substitution method. Meanwhile, for the integrity of the paper, we introduce a new model, the sparse group lasso support vector machine (SGL-SVM), and apply it to music information retrieval. Theoretical analysis confirms that the computational complexity of the proposed algorithm is not affected by different regularization terms and loss functions, highlighting the universality of the parallel algorithm. Experiments on synthetic and free music archiv datasets demonstrate the reliability, stability, and efficiency of the algorithm.

Paper Structure

This paper contains 18 sections, 52 equations, 5 figures, 8 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. SGL-SVM
  4. Some extensions for CR-SVMs
  5. Parallel ADMM algorithm for CR-SVMs
  6. Parallel ADMM algorithm for CR-SVMs
  7. For the problem (\ref{['iteration-betak']})
  8. For the problem (\ref{['iteration-theta']})
  9. For the problem (\ref{['iteration-xi']})
  10. For the problem (\ref{['iteration-beta']})
  11. For the problem (\ref{['iteration-beta0']})
  12. Gaussian Back Substitution
  13. Convergence and computational cost analysis
  14. Numerical results
  15. Synthetic Data
  16. ...and 3 more sections

Figures (5)

  • Figure 1: Schematic diagram of the implementation of parallel ADMM algorithm
  • Figure 2: Parallel ADMM convergence performance with different numbers of sub-machines.
  • Figure 3: A schematic diagram depicting the variations of computation time and classification accuracy rate of four parallel algorithms with the number of sub-machines.
  • Figure 4: Group weight norms under different penalty parameters.
  • Figure 5: Feature group weight distribution.