Feature graphs for interpretable unsupervised tree ensembles: centrality, interaction, and application in disease subtyping

TL;DR

The paper addresses the challenge of interpretability in unsupervised learning for disease subtyping by introducing feature graphs derived from unsupervised random forests. It defines four edge-building criteria and two graph-mining strategies (brute-force and greedy) to identify central features and discriminative feature pairs, with cluster-specific graph variants to highlight per-cluster relevance. Extensive evaluation on synthetic and benchmark datasets shows that features with high centrality and heavy inter-feature edges align with true relevance and discriminative power, while the greedy method offers scalable, monotonic improvements over impurity-based baselines. A kidney cancer case study demonstrates the practical utility of the approach in revealing cluster-specific gene interactions and survival differences, supporting its potential for interpretable biomedical clustering and disease subtyping.

Abstract

Interpretable machine learning has emerged as central in leveraging artificial intelligence within high-stakes domains such as healthcare, where understanding the rationale behind model predictions is as critical as achieving high predictive accuracy. In this context, feature selection assumes a pivotal role in enhancing model interpretability by identifying the most important input features in black-box models. While random forests are frequently used in biomedicine for their remarkable performance on tabular datasets, the accuracy gained from aggregating decision trees comes at the expense of interpretability. Consequently, feature selection for enhancing interpretability in random forests has been extensively explored in supervised settings. However, its investigation in the unsupervised regime remains notably limited. To address this gap, the study introduces novel methods to construct feature graphs from unsupervised random forests and feature selection strategies to derive effective feature combinations from these graphs. Feature graphs are constructed for the entire dataset as well as individual clusters leveraging the parent-child node splits within the trees, such that feature centrality captures their relevance to the clustering task, while edge weights reflect the discriminating power of feature pairs. Graph-based feature selection methods are extensively evaluated on synthetic and benchmark datasets both in terms of their ability to reduce dimensionality while improving clustering performance, as well as to enhance model interpretability. An application on omics data for disease subtyping identifies the top features for each cluster, showcasing the potential of the proposed approach to enhance interpretability in clustering analyses and its utility in a real-world biomedical application.

Figures (8)

• Figure 1: Feature pairs discriminating (a) 1, (b) 2, (c) 3 and (d) 4 clusters.
• Figure 2: Evaluating out-degree centrality and edge weights of the feature graphs.
• Figure 3: Evaluating cluster-specific feature graphs.
• Figure 4: Evaluating brute force and greedy feature selection approaches.
• Figure 5: Evaluating feature selection strategies in case of redundant features.