Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Iterated Relevance Matrix Analysis (IRMA) for the identification of class-discriminative subspaces

Sofie Lövdal, Michael Biehl

TL;DR

The IRMA-based class-discriminative subspace can be used for dimensionality reduction and the training of robust classifiers with potentially improved performance and enables improved low-dimensional representations and visualizations of labeled data sets.

Abstract

We introduce and investigate the iterated application of Generalized Matrix Learning Vector Quantizaton for the analysis of feature relevances in classification problems, as well as for the construction of class-discriminative subspaces. The suggested Iterated Relevance Matrix Analysis (IRMA) identifies a linear subspace representing the classification specific information of the considered data sets using Generalized Matrix Learning Vector Quantization (GMLVQ). By iteratively determining a new discriminative subspace while projecting out all previously identified ones, a combined subspace carrying all class-specific information can be found. This facilitates a detailed analysis of feature relevances, and enables improved low-dimensional representations and visualizations of labeled data sets. Additionally, the IRMA-based class-discriminative subspace can be used for dimensionality reduction and the training of robust classifiers with potentially improved performance.

Iterated Relevance Matrix Analysis (IRMA) for the identification of class-discriminative subspaces

TL;DR

The IRMA-based class-discriminative subspace can be used for dimensionality reduction and the training of robust classifiers with potentially improved performance and enables improved low-dimensional representations and visualizations of labeled data sets.

Abstract

We introduce and investigate the iterated application of Generalized Matrix Learning Vector Quantizaton for the analysis of feature relevances in classification problems, as well as for the construction of class-discriminative subspaces. The suggested Iterated Relevance Matrix Analysis (IRMA) identifies a linear subspace representing the classification specific information of the considered data sets using Generalized Matrix Learning Vector Quantization (GMLVQ). By iteratively determining a new discriminative subspace while projecting out all previously identified ones, a combined subspace carrying all class-specific information can be found. This facilitates a detailed analysis of feature relevances, and enables improved low-dimensional representations and visualizations of labeled data sets. Additionally, the IRMA-based class-discriminative subspace can be used for dimensionality reduction and the training of robust classifiers with potentially improved performance.
Paper Structure (10 sections, 8 equations, 5 figures, 1 table)

This paper contains 10 sections, 8 equations, 5 figures, 1 table.

Table of Contents

  1. Introduction
  2. Methods
  3. Iterated Relevance Matrix Learning
  4. Experiments
  5. Results and Discussion
  6. Artificial Data
  7. Wisconsin Diagnostic Breast Cancer data
  8. Segmentation data
  9. Conclusion and Outlook
  10. Acknowledgements

Figures (5)

  • Figure 1: Artificial data: original features $x_1, x_2$ of the data set (a), projections on $v_1^{(0)}$, $v_2^{(0)}$ of unrestricted GMLVQ (b), and projections on the eigenvectors $v_1^{(0)}$ and $v_1^{(1)}$ of the unrestricted system and the first iteration of IRMA in (c).
  • Figure 2: Wisconsin data set: Projections after 0th (a), 1st iteration (b), and data projected onto leading eigenvectors of 0th and 1st iteration, respectively (c).
  • Figure 3: Wisconsin data set: Diagonal of $\Lambda$ per iteration $(i)$, which is indicated as $i$ in the upper left corner of each panel. In addition, the obtained random sampling validation $BAC$ w.r.t. test data are shown.
  • Figure 4: Segmentation data set: Diagonal of $\Lambda$ per iteration $(i)$, which is indicated as $i$in each panel, where IRMA has been applied using all available data. Three eigenvectors are removed per iteration for this seven-class problem, and the average $BAC$ w.r.t. test data is indicated on top of each panel.
  • Figure 5: Segmentation data set: Projections after 0th (a), 1st iteration (b) and 2nd iteration (c). The main cluster was zoomed in on in (c), cutting out a few outliers.