Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Can an unsupervised clustering algorithm reproduce a categorization system?

Nathalia Castellanos, Dhruv Desai, Sebastian Frank, Stefano Pasquali, Dhagash Mehta

TL;DR

This paper investigates whether unsupervised clustering can reproduce expert-created categorization systems in finance, showing that success depends on feature predictive power and the chosen distance metric. It compares baseline K-means with Euclidean distance to supervised distance-learning approaches—Mahalanobis metric learning and RF-PHATE—on toy datasets and Morningstar fund categorization, incorporating $d_M(x,y)=\sqrt{(x-y)^T M (x-y)}$ for the Mahalanobis distance. The results reveal that pure unsupervised clustering often fails to recover ground-truth classes, but learning distance metrics from labels and using RF-PHATE can align clusters with ground-truth categories, frequently across external metrics; RF-PHATE, in particular, shows strong performance on mixed-type data and robustness to missing values. The work underscores that evaluating categorization consistency should consider supervised metric-learning baselines, and it highlights the limited reliability of standard internal clustering metrics for identifying optimal cluster counts.

Abstract

Peer analysis is a critical component of investment management, often relying on expert-provided categorization systems. These systems' consistency is questioned when they do not align with cohorts from unsupervised clustering algorithms optimized for various metrics. We investigate whether unsupervised clustering can reproduce ground truth classes in a labeled dataset, showing that success depends on feature selection and the chosen distance metric. Using toy datasets and fund categorization as real-world examples we demonstrate that accurately reproducing ground truth classes is challenging. We also highlight the limitations of standard clustering evaluation metrics in identifying the optimal number of clusters relative to the ground truth classes. We then show that if appropriate features are available in the dataset, and a proper distance metric is known (e.g., using a supervised Random Forest-based distance metric learning method), then an unsupervised clustering can indeed reproduce the ground truth classes as distinct clusters.

Can an unsupervised clustering algorithm reproduce a categorization system?

TL;DR

This paper investigates whether unsupervised clustering can reproduce expert-created categorization systems in finance, showing that success depends on feature predictive power and the chosen distance metric. It compares baseline K-means with Euclidean distance to supervised distance-learning approaches—Mahalanobis metric learning and RF-PHATE—on toy datasets and Morningstar fund categorization, incorporating for the Mahalanobis distance. The results reveal that pure unsupervised clustering often fails to recover ground-truth classes, but learning distance metrics from labels and using RF-PHATE can align clusters with ground-truth categories, frequently across external metrics; RF-PHATE, in particular, shows strong performance on mixed-type data and robustness to missing values. The work underscores that evaluating categorization consistency should consider supervised metric-learning baselines, and it highlights the limited reliability of standard internal clustering metrics for identifying optimal cluster counts.

Abstract

Peer analysis is a critical component of investment management, often relying on expert-provided categorization systems. These systems' consistency is questioned when they do not align with cohorts from unsupervised clustering algorithms optimized for various metrics. We investigate whether unsupervised clustering can reproduce ground truth classes in a labeled dataset, showing that success depends on feature selection and the chosen distance metric. Using toy datasets and fund categorization as real-world examples we demonstrate that accurately reproducing ground truth classes is challenging. We also highlight the limitations of standard clustering evaluation metrics in identifying the optimal number of clusters relative to the ground truth classes. We then show that if appropriate features are available in the dataset, and a proper distance metric is known (e.g., using a supervised Random Forest-based distance metric learning method), then an unsupervised clustering can indeed reproduce the ground truth classes as distinct clusters.
Paper Structure (22 sections, 4 equations, 2 figures, 5 tables)

This paper contains 22 sections, 4 equations, 2 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Previous Works and Our Contribution
  3. Methodology
  4. K-means Clustering:
  5. Supervised Clustering using Mahalanobis Metric
  6. Supervised Clustering using RF-PHATE
  7. Multidimensional Scaling (MDS)
  8. Evaluation Metrics
  9. Evaluating Random Forest Training
  10. Evaluating K-means Clustering
  11. Evaluating How Well Clustering Reproduces Ground Truth Classes
  12. Data Description
  13. Data Preparation
  14. Computational Details
  15. Results
  16. ...and 7 more sections

Figures (2)

  • Figure 1: Plots comparing optimal number clusters $K_C$ based on different evaluation metrics and the number of ground truth classes (horizontal line). Further explanation of the figures is provided in Section \ref{['sec:results']}.
  • Figure 2: MDS Plots for Public datasets and Funds data using three different distance metrics. The colors represent different ground-truth classes.