LHT: Statistically-Driven Oblique Decision Trees for Interpretable Classification

TL;DR

This paper addresses the limited expressive power and interpretability of axis-parallel trees by introducing Learning Hyperplane Tree (LHT), a non-iterative, statistically-driven oblique tree that builds hyperplanes from feature expectation differences within each node and uses a locally fitted piecewise linear membership function at leaves. The approach yields $y(x) = FS(x) - c$ splits with feature weights $w_i = SD_i / \overline{SD}$ and a complexity bound of $O(mnd)$, along with a universal approximation property for continuous functions on compact sets. It also extends to an LH Forest for improved robustness and provides explicit interpretability through per-split feature contributions. Empirical results on diverse benchmarks show competitive accuracy against SOTA oblique trees and gradient-boosted ensembles, while maintaining deterministic, transparent decision rules and scalable training. The work offers a practical, theoretically grounded alternative for interpretable classification on tabular data, with code available at the referenced GitHub repository.

Abstract

We introduce the Learning Hyperplane Tree (LHT), a novel oblique decision tree model designed for expressive and interpretable classification. LHT fundamentally distinguishes itself through a non-iterative, statistically-driven approach to constructing splitting hyperplanes. Unlike methods that rely on iterative optimization or heuristics, LHT directly computes the hyperplane parameters, which are derived from feature weights based on the differences in feature expectations between classes within each node. This deterministic mechanism enables a direct and well-defined hyperplane construction process. Predictions leverage a unique piecewise linear membership function within leaf nodes, obtained via local least-squares fitting. We formally analyze the convergence of the LHT splitting process, ensuring that each split yields meaningful, non-empty partitions. Furthermore, we establish that the time complexity for building an LHT up to depth $d$ is $O(mnd)$, demonstrating the practical feasibility of constructing trees with powerful oblique splits using this methodology. The explicit feature weighting at each split provides inherent interpretability. Experimental results on benchmark datasets demonstrate LHT's competitive accuracy, positioning it as a practical, theoretically grounded, and interpretable alternative in the landscape of tree-based models. The implementation of the proposed method is available at https://github.com/Hongyi-Li-sz/LHT_model.

Figures (6)

• Figure 1: The structure of LHT is illustrated. LHT consists of two types of blocks: branching blocks, which employ hyperplanes for sample partitioning, and leaf blocks, where least-squares fitted membership functions are used for classifying test samples. The construction of the LHT hyperplane consists of feature selection and block splitting.
• Figure 2: The case where $c=\min \text{TFS}$ is illustrated when $N_3 = N_{\max}$ and $N_{max}\geq\gamma$, where $c = \min \text{TFS}$. Samples with a feature-weighted sum smaller than $c$ are assigned to the left subblock 1, and the remaining samples are assigned to the right subblock 2. Since all samples in left subblock 1 are non-target class samples, it is marked as a leaf block. Right subblock 2 still contains mixed samples, and the allocation process continues based on the data within the block until all samples are properly classified.
• Figure 3: Illustrative cases of pure block generation when $N_{\max} \geq \gamma$. Each subfigure depicts the resulting subblocks for different $c$ values.
• Figure 4: The LHT structures of the three classes in the Wine dataset are shown, with the left side corresponding to the case where $\beta=0$, and the right side to the case where $\beta=0.25$.
• Figure 5: Visualization of the feature weights for each branching block of the three LHTs corresponding to the three classes in the Wine dataset ($\beta=0$).

Theorems & Definitions (9)

• Theorem 1: Splitting Guarantee
• Theorem 2: Time Complexity
• Theorem 3
• Theorem 1
• proof
• Theorem 2
• proof
• Theorem 3
• proof