RAZOR: Refining Accuracy by Zeroing Out Redundancies

TL;DR

RAZOR is a novel instance selection technique designed to extract a significantly smaller yet sufficiently informative subset from a larger set of instances without compromising the learning process, suitable for large-scale datasets.

Abstract

In many application domains, the proliferation of sensors and devices is generating vast volumes of data, imposing significant pressure on existing data analysis and data mining techniques. Nevertheless, an increase in data volume does not inherently imply an increase in informational content, as a substantial portion may be redundant or represent noise. This challenge is particularly evident in the deep learning domain, where the utility of additional data is contingent on its informativeness. In the absence of such, larger datasets merely exacerbate the computational cost and complexity of the learning process. To address these challenges, we propose RAZOR, a novel instance selection technique designed to extract a significantly smaller yet sufficiently informative subset from a larger set of instances without compromising the learning process. RAZOR has been specifically engineered to be robust, efficient, and scalable, making it suitable for large-scale datasets. Unlike many techniques in the literature, RAZOR is capable of operating in both supervised and unsupervised settings. Experimental results demonstrate that RAZOR outperforms recent state-of-the-art techniques in terms of both effectiveness and efficiency.

Figures (3)

• Figure 1: Overview of RAZOR's instance selection algorithm based on entropy clustering. The process includes several key steps: (a) Data Projection: The original training set $\Omega$ is processed through a feature extractor to obtain feature vectors $v_i = f(\omega_i)$, which are then mapped into a feature space $\mathcal{V}$; (b) Initial Partitioning: The dataset $\mathcal{V}$ is partitioned using k-means clustering into initial clusters to ensure manageability; (c) Split & Merge Process: An iterative process involving entropy-based splitting and KNN-based merging is employed to refine the clusters. The entropy-based split phase identifies representative samples while the KNN-based merge phase consolidates similar clusters; (d) Final Aggregation: The final set of clusters is obtained merging clusters according to the shape of the corresponding convex hulls; (e) Sample Selection: The convex hull of each final cluster is analyzed to identify the most distinctive and representative points, which are then selected for the final training set. Note that steps b-e do not involve the original objects in $\Omega$ but are carried out directly in $\mathcal{V}$. In (c) and (e), the parallel with the original objects is reported solely to clarify the processing procedure.
• Figure 2: The performance of the RAZOR algorithm, as measured by Adjusted Mutual Information (AMI). Six subplots depict different VLEN (vector length) values. The AMI values are plotted against the number of clusters (NClusters) and the number of points per cluster (NPoints). Each subplot provides a color-coded AMI gradient, with yellow representing higher AMI values and blue representing lower values.
• Figure 3: The execution times and number of iterations required by the Iterative Entropy Clusters Algorithm under varying conditions, with results shown for single-core (left) and four-core (right) processing. The three axes represent the number of points per cluster, the number of clusters, and the vector length, respectively. The size of each point represents the number of iterations required to converge (from 1 to $maxiter$), while the brightness of the points corresponds to the execution time, which is also displayed numerically next to the points in seconds.