CART-ELC: Oblique Decision Tree Induction via Exhaustive Search
Andrew D. Laack
TL;DR
CART-ELC introduces an exact oblique-tree induction method that exhaustively searches a restricted hyperplane family, controlled by a single parameter $r$ that limits the number of non-zero coefficients and the samples a hyperplane must pass through. The algorithm uses a choice of splitting criteria (Twoing, Gini, Information Gain) and provides a rigorous time-complexity analysis showing feasibility on small datasets for modest $r$ while acknowledging scalability challenges. Empirical results on six varied, small-to-moderate datasets show CART-ELC delivers competitive accuracies and often smaller, more interpretable trees compared to existing oblique and axis-aligned methods, with statistical analysis favoring CART-ELC in several domains. The work highlights a path toward interpretable oblique splits and suggests future integration with ensemble methods to mitigate computational costs while retaining accuracy benefits.
Abstract
Oblique decision trees have attracted attention due to their potential for improved classification performance over traditional axis-aligned decision trees. However, methods that rely on exhaustive search to find oblique splits face computational challenges. As a result, they have not been widely explored. We introduce a novel algorithm, Classification and Regression Tree - Exhaustive Linear Combinations (CART-ELC), for inducing oblique decision trees that performs an exhaustive search on a restricted set of hyperplanes. We then investigate the algorithm's computational complexity and its predictive capabilities. Our results demonstrate that CART-ELC consistently achieves competitive performance on small datasets, often yielding statistically significant improvements in classification accuracy relative to existing decision tree induction algorithms, while frequently producing shallower, simpler, and thus more interpretable trees.