Cultivating Archipelago of Forests: Evolving Robust Decision Trees through Island Coevolution

TL;DR

This work tackles robustness of decision-tree ensembles to adversarial perturbations by formulating a robust forest objective under $L_\infty$ perturbations and solving it with an island-based coevolutionary algorithm, ICoEvoRDF. The method evolves multiple populations of robust trees and perturbations across interconnected islands, periodically migrating top performers and weighting trees via Mixed Nash Equilibrium to form a robust ensemble. Key contributions include the first integration of multi-population island models with MNE-based voting for robust decision forests, comprehensive ablations, and strong empirical gains across 20 benchmarks in both adversarial accuracy and minimax regret. The approach offers a flexible, interpretable framework that can incorporate trees from diverse origins and suggests future work on fairness and explainability in robust ML systems.

Abstract

Decision trees are widely used in machine learning due to their simplicity and interpretability, but they often lack robustness to adversarial attacks and data perturbations. The paper proposes a novel island-based coevolutionary algorithm (ICoEvoRDF) for constructing robust decision tree ensembles. The algorithm operates on multiple islands, each containing populations of decision trees and adversarial perturbations. The populations on each island evolve independently, with periodic migration of top-performing decision trees between islands. This approach fosters diversity and enhances the exploration of the solution space, leading to more robust and accurate decision tree ensembles. ICoEvoRDF utilizes a popular game theory concept of mixed Nash equilibrium for ensemble weighting, which further leads to improvement in results. ICoEvoRDF is evaluated on 20 benchmark datasets, demonstrating its superior performance compared to state-of-the-art methods in optimizing both adversarial accuracy and minimax regret. The flexibility of ICoEvoRDF allows for the integration of decision trees from various existing methods, providing a unified framework for combining diverse solutions. Our approach offers a promising direction for developing robust and interpretable machine learning models

Figures (1)

• Figure 1: The ICoEvoRDF scheme. The illustration shows 4 islands with ring migration topology. Each island contains two populations: DTs and perturbations which are developed alternately using evolutionary operators.