Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

GPT-HTree: A Decision Tree Framework Integrating Hierarchical Clustering and Large Language Models for Explainable Classification

Te Pei, Fuat Alican, Aaron Ontoyin Yin, Yigit Ihlamur

TL;DR

GPT-HTree presents an explainable classification framework that integrates hierarchical clustering, cluster-specific decision trees, and large language models to model heterogeneous founder populations in venture capital. By combining CTGAN-based resampling, eight emergent founder personas, and LLM generated persona descriptions, the approach yields localized predictive rules with interpretable narratives, bridging quantitative patterns and practical insights. The method demonstrates improved discernment of high-potential founder clusters and provides actionable factors driving success, enhancing deal sourcing and mentorship decisions. This work advances interpretable ML by operationalizing LLMs to describe clusters and integrate narrative guidance into structured predictive models, with potential applications beyond venture capital.

Abstract

This paper introduces GPT-HTree, a framework combining hierarchical clustering, decision trees, and large language models (LLMs) to address this challenge. By leveraging hierarchical clustering to segment individuals based on salient features, resampling techniques to balance class distributions, and decision trees to tailor classification paths within each cluster, GPT-HTree ensures both accuracy and interpretability. LLMs enhance the framework by generating human-readable cluster descriptions, bridging quantitative analysis with actionable insights.

GPT-HTree: A Decision Tree Framework Integrating Hierarchical Clustering and Large Language Models for Explainable Classification

TL;DR

GPT-HTree presents an explainable classification framework that integrates hierarchical clustering, cluster-specific decision trees, and large language models to model heterogeneous founder populations in venture capital. By combining CTGAN-based resampling, eight emergent founder personas, and LLM generated persona descriptions, the approach yields localized predictive rules with interpretable narratives, bridging quantitative patterns and practical insights. The method demonstrates improved discernment of high-potential founder clusters and provides actionable factors driving success, enhancing deal sourcing and mentorship decisions. This work advances interpretable ML by operationalizing LLMs to describe clusters and integrate narrative guidance into structured predictive models, with potential applications beyond venture capital.

Abstract

This paper introduces GPT-HTree, a framework combining hierarchical clustering, decision trees, and large language models (LLMs) to address this challenge. By leveraging hierarchical clustering to segment individuals based on salient features, resampling techniques to balance class distributions, and decision trees to tailor classification paths within each cluster, GPT-HTree ensures both accuracy and interpretability. LLMs enhance the framework by generating human-readable cluster descriptions, bridging quantitative analysis with actionable insights.
Paper Structure (34 sections, 5 equations, 7 figures, 4 tables, 3 algorithms)

This paper contains 34 sections, 5 equations, 7 figures, 4 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Motivation
  3. Empirical Context and Challenges in Venture Capital
  4. Key Contributions
  5. Related Work
  6. Decision Trees, Ensemble Methods, and Their Limitations
  7. Clustering-Based Classification and Hybrid Approaches
  8. Large Language Models in Machine Learning
  9. Methodology
  10. Data Preprocessing
  11. Resampling Analysis
  12. Hierarchical Clustering
  13. Cluster Analysis and LLM Interpretation
  14. Persona Characterization with Large Language Models
  15. Decision Tree Classification within Clusters
  16. ...and 19 more sections

Figures (7)

  • Figure 1: An overview of the Machine Learning Workflow, where task specification serves as the input, encompassing key stages of data and feature engineering, model selection and hyperparameter optimization, and workflow evaluation. Figure reproduced from gu2024large.
  • Figure 2: Example of LLM-Generated Persona Descriptions.
  • Figure 3: LLM Prompt Design for Founder Persona Generation
  • Figure 4: Resampling Techniques and Their Impact on Class Balance.
  • Figure 5: Main Cluster after key characteristics mining
  • ...and 2 more figures