Classy Ensemble: A Novel Ensemble Algorithm for Classification

TL;DR

Classy Ensemble introduces a per-class accuracy-based, weighted ensemble method for classification that selects top-$k$ models for each class and aggregates their outputs with class-aware voting. The approach is benchmarked against order-based pruning, cluster-based pruning, and Lexigarden across 153 PMLB datasets, showing statistically significant improvements on many datasets and competitive performance on large-scale DL tasks. Extensions include Classy Cluster Ensemble and an Evolutionary Ensemble that searches for effective model subsets, with notable gains on ImageNet when combined with DL models. The work demonstrates a scalable, straightforward ensemble paradigm that yields practical accuracy gains and invites further refinement with class-dependent weighting and broader DL integration.

Abstract

We present Classy Ensemble, a novel ensemble-generation algorithm for classification tasks, which aggregates models through a weighted combination of per-class accuracy. Tested over 153 machine learning datasets we demonstrate that Classy Ensemble outperforms two other well-known aggregation algorithms -- order-based pruning and clustering-based pruning -- as well as the recently introduced lexigarden ensemble generator. We then present three enhancements: 1) Classy Cluster Ensemble, which combines Classy Ensemble and cluster-based pruning; 2) Deep Learning experiments, showing the merits of Classy Ensemble over four image datasets: Fashion MNIST, CIFAR10, CIFAR100, and ImageNet; and 3) Classy Evolutionary Ensemble, wherein an evolutionary algorithm is used to select the set of models which Classy Ensemble picks from. This latter, combining learning and evolution, resulted in improved performance on the hardest dataset.