Multilabel Prototype Generation for Data Reduction in k-Nearest Neighbour classification

TL;DR

This work addresses the data reduction bottleneck of $k$NN in multilabel settings by adapting four multiclass Prototype Generation strategies to multilabel data. It introduces MChen, MRSP1, MRSP2, and MRSP3, integrates them with three multilabel $k$NN classifiers, and evaluates on 12 multilabel corpora under varying label-noise levels. The results show these methods often improve both reduction and classification performance over the existing MRHC approach and the ALL baseline, with notable robustness to noise and configurable trade-offs between efficiency and efficacy. The proposed framework broadens the practical applicability of PG in multilabel scenarios and suggests directions for improving robustness to class imbalance and computational efficiency in real-world tasks.

Abstract

Prototype Generation (PG) methods are typically considered for improving the efficiency of the $k$-Nearest Neighbour ($k$NN) classifier when tackling high-size corpora. Such approaches aim at generating a reduced version of the corpus without decreasing the classification performance when compared to the initial set. Despite their large application in multiclass scenarios, very few works have addressed the proposal of PG methods for the multilabel space. In this regard, this work presents the novel adaptation of four multiclass PG strategies to the multilabel case. These proposals are evaluated with three multilabel $k$NN-based classifiers, 12 corpora comprising a varied range of domains and corpus sizes, and different noise scenarios artificially induced in the data. The results obtained show that the proposed adaptations are capable of significantly improving -- both in terms of efficiency and classification performance -- the only reference multilabel PG work in the literature as well as the case in which no PG method is applied, also presenting a statistically superior robustness in noisy scenarios. Moreover, these novel PG strategies allow prioritising either the efficiency or efficacy criteria through its configuration depending on the target scenario, hence covering a wide area in the solution space not previously filled by other works.

Figures (6)

• Figure 1: Graphical illustration of the multiclass Chen PG method. The example depicts the results of the space partitioning process (cases \ref{['fig:chen1']} to \ref{['fig:chen3']}) and the prototype merging phase (case \ref{['fig:chen4']}) when considering $n_d=3$ subsets. Symbols $p_{1}$ and $p_{2}$ denote the two furthest prototypes in the cluster to be divided.
• Figure 2: Graphical illustration of the multilabel PG proposals introduced in the work. Figure \ref{['fig:ml_initial_space']} represents a multilabel set of train data $\mathcal{T}_{ml}$ to be reduced. Figure \ref{['fig:ml_space_part_result']} shows the space partitioning results on which the different reduction proposals are based, except for the MRSP3 one, whose case is illustrated in Figure \ref{['fig:mrsp3_part']}. Prototype merging graphs \ref{['fig:mchen']} and \ref{['fig:mrsp3_merging']} depict the number of prototypes---denoted as #prot---and the cardinality of labels---$\#\square$, $\#\circ$, and $\#\diamond$---for each of the original clusters.
• Figure 3: Experimental scheme for the comparative assessment of the PG methods.
• Figure 4: Results in terms of HL and resulting size obtained with the $k$NN-based classifiers when considering the PG methods and the exhaustive search case (ALL) for the different $k$ values tested. Circled methods and dashed lines represent the non-dominated elements and the Pareto frontiers in each scenario, respectively. For easier comparison, shaded areas depict the regions in the solution space occupied by the baseline cases (MRHC and ALL).
• Figure 5: Results in terms of HL and resulting size for the different noise scenarios when considering the PG methods and the exhaustive search case (ALL) for the $k$ values tested. Note that each sample constitutes the average performance obtained for the three classification methods studied. Circled methods and dashed lines represent the non-dominated elements and the Pareto frontiers in each scenario, respectively. For easier comparison, shaded areas depict the regions in the solution space occupied by the baseline cases (MRHC and ALL).