DRtool: An Interactive Tool for Analyzing High-Dimensional Clusterings

Abstract

When faced with new data, we often conduct a cluster analysis to obtain a better understanding of the data's structure and the archetypical samples present in the data. This process often includes visualization of the data, either as a way to discover or verify clusters. However, the increases in data complexity and dimensionality has made this step very tricky. To visualize data, nonlinear dimension reduction methods are the de facto standard for their ability to non-uniformly stretch and shrink space in order to preserve local clusters. Because this process requires a drastic manipulation of space, however, nonlinear dimension reduction methods are known to produce false structures, especially when mishandled. A common consequence that often goes undetected by the untrained eye is over-clustering of the data. In efforts to deal with this phenomenon, we developed an interactive tool that empowers analysts to distinguish false clusters and better interpret their high-dimensional clustering results. The tool uses various analytical plots to provide a multi-faceted perspective on the data's global structure as well as local inter-cluster relationships, helping users determine the legitimacy of their high-dimensional clustering results. The tool is available via an R package named DRtool.

Figures (14)

• Figure 1: Example of Algorithm \ref{['alg1']}. The first step is to compute the medoid subtree $T'$. Then non-medoid degree-2 vertices are replaced with single edges.
• Figure 2: Example of Algorithm \ref{['alg3']}. First, degree-2 vertices belonging to neither group are replaced with single edges. Then adjacent vertices belonging to neither group are collapsed into a single vertex.
• Figure 3: MST stability results on the MNIST data set.
• Figure 4: Power analysis of the MST test.
• Figure 5: UMAP embedding of the MNIST data set colored according to k-means clustering.