Trinary Decision Trees for handling missing data

TL;DR

The paper introduces the Trinary decision tree to improve missing data handling in regression and classification trees by routing missing values to a dedicated third node that uses the full node data for its parameter estimate, preserving interpretability and yielding locally unbiased estimates. The split objective includes the left, right, and missing branches, ensuring missing data do not contaminate standard node estimates, with theoretical bias results showing unbiasedness for the Trinary estimator under MCAR. A hybrid TrinaryMIA variant combines Trinary and Missing In Attributes strategies to deliver robust performance across MCAR and IM missingness. Numerical experiments across diverse datasets reveal strong MCAR performance for Trinary/TrinaryMIA and robust IM performance for TrinaryMIA, albeit with slower training, suggesting potential as a weak learner in ensemble methods like GBMs.

Abstract

This paper introduces the Trinary decision tree, an algorithm designed to improve the handling of missing data in decision tree regressors and classifiers. Unlike other approaches, the Trinary decision tree does not assume that missing values contain any information about the response. Both theoretical calculations on estimator bias and numerical illustrations using real data sets are presented to compare its performance with established algorithms in different missing data scenarios (Missing Completely at Random (MCAR), and Informative Missingness (IM)). Notably, the Trinary tree outperforms its peers in MCAR settings, especially when data is only missing out-of-sample, while lacking behind in IM settings. A hybrid model, the TrinaryMIA tree, which combines the Trinary tree and the Missing In Attributes (MIA) approach, shows robust performance in all types of missingness. Despite the potential drawback of slower training speed, the Trinary tree offers a promising and more accurate method of handling missing data in decision tree algorithms.

Figures (2)

• Figure 1: Visualization of a Trinary tree with depth $1$ for a covariate with $p=2$ dimensions. Note that since the third node is considered to be at the same depth level as the root node, an additional split is made. Since the best performing split covariate $j=1$ is no longer available, the second-best split covariate $j=2$ is used for the second split. Since that covariate could also be missing, the third node has its own third daughter node. Since there are no further covariates to split on, that node is a terminal node.
• Figure 2: Average excess loss per missingness ratio for the tree algorithms in different kinds of missingness on all data sets