The Eigenvalues Entropy as a Classifier Evaluation Measure

TL;DR

This work introduces eigenvalues entropy (EVE) as a novel classifier evaluation measure derived from a transformed confusion matrix. By converting the confusion matrix into a column-stochastic form $ extbf{P}$ and a symmetric matrix $ extbf{B} = ( extbf{P} + extbf{P}^{ op})/2$, it defines $eve( extbf{B}) = -rac{1}{ obreak obreak} rac{ heta_i}{n^*} rac{ ext{log}( heta_i/n^*)}{ ext{log}(n)}$, where $ heta_i$ are the positive eigenvalues and $n^*$ their sum, yielding $eve( extbf{B}) o 1$ for an ideal classifier. The paper derives binary-case relations linking eigenvalues to $AUC$ and the Gini index, establishes Gershgorin-based bounds for $ extbf{B}$’s eigenvalues, and even provides a method to estimate the confusion matrix to mitigate class imbalance. Extensive experiments on binary and multi-class datasets, including MNIST, show EVE often outperforms traditional measures and remains robust under different CM representations. The approach offers a theoretically grounded, practically impactful tool for evaluating classifiers and comparing clustering or rater agreements, particularly when class imbalance is present.

Abstract

Classification is a machine learning method used in many practical applications: text mining, handwritten character recognition, face recognition, pattern classification, scene labeling, computer vision, natural langage processing. A classifier prediction results and training set information are often used to get a contingency table which is used to quantify the method quality through an evaluation measure. Such measure, typically a numerical value, allows to choose a suitable method among several. Many evaluation measures available in the literature are less accurate for a dataset with imbalanced classes. In this paper, the eigenvalues entropy is used as an evaluation measure for a binary or a multi-class problem. For a binary problem, relations are given between the eigenvalues and some commonly used measures, the sensitivity, the specificity, the area under the operating receiver characteristic curve and the Gini index. A by-product result of this paper is an estimate of the confusion matrix to deal with the curse of the imbalanced classes. Various data examples are used to show the better performance of the proposed evaluation measure over the gold standard measures available in the literature.

Figures (2)

• Figure 1: Gershgorin circles
• Figure 2: Behavior of $\lambda_2$ when $p_{11}$ and $p_{22}$ vary

Theorems & Definitions (5)

• proof
• proof
• proof
• proof
• Conjecture 1