On the Robustness of Random Forest Against Untargeted Data Poisoning: An Ensemble-Based Approach

TL;DR

This paper investigates how to bolster random forests against untargeted data poisoning by introducing a hash-based ensemble defense. By partitioning the training data into $N$ disjoint subsets via hashing and training one RF per partition with majority voting at inference, the approach aims to reduce the influence of poisoned points while maintaining practicality. Across multiple tabular, binary classification datasets, label flipping emerges as the most harmful perturbation to a monolithic RF, yet the ensemble consistently improves robustness and sustains performance within reasonable resource bounds; larger ensembles yield stronger protection, particularly for larger poisoning levels. The study demonstrates a practical, scalable defense that leverages model strengthening through ensembling, offering nuanced guidance on ensemble size and resource trade-offs for real-world deployments.

Abstract

Machine learning is becoming ubiquitous. From finance to medicine, machine learning models are boosting decision-making processes and even outperforming humans in some tasks. This huge progress in terms of prediction quality does not however find a counterpart in the security of such models and corresponding predictions, where perturbations of fractions of the training set (poisoning) can seriously undermine the model accuracy. Research on poisoning attacks and defenses received increasing attention in the last decade, leading to several promising solutions aiming to increase the robustness of machine learning. Among them, ensemble-based defenses, where different models are trained on portions of the training set and their predictions are then aggregated, provide strong theoretical guarantees at the price of a linear overhead. Surprisingly, ensemble-based defenses, which do not pose any restrictions on the base model, have not been applied to increase the robustness of random forest models. The work in this paper aims to fill in this gap by designing and implementing a novel hash-based ensemble approach that protects random forest against untargeted, random poisoning attacks. An extensive experimental evaluation measures the performance of our approach against a variety of attacks, as well as its sustainability in terms of resource consumption and performance, and compares it with a traditional monolithic model based on random forest. A final discussion presents our main findings and compares our approach with existing poisoning defenses targeting random forests.

Figures (8)

• Figure 1: Overview of our hash-based ensemble approach.
• Figure 2: Evaluation process based on the ensemble approach in Figure \ref{['fig:ensembleapp']}.
• Figure 3: Results for label flipping with monolithic ($N$$=$$1$) and the smallest ($N$$=$$3$) and largest ($N$$=$$21$) ensemble models for datasets M2, AM, SB, and DR.
• Figure 4: Results for other attacks averaged over $\epsilon_{\text{p}}$ with perturbations zeroing, noising, and out-/of-/ranging abbreviated as zero, noise, and OoR, respectively.
• Figure 5: Execution time for model training varying $N$ and datasets M2, AM, SB, and DR.