Stop Misusing t-SNE and UMAP for Visual Analytics

TL;DR

This paper investigates the widespread misuse of t-SNE and UMAP in visual analytics, where local-neighborhood preservation is overextended to infer global cluster relationships. Through a literature review of 136 papers, interviews with 12 practitioners, and interviews with 8 DR experts, the authors identify limited DR literacy and motivational gaps as primary drivers of misuse, with existing mitigation efforts largely ineffective. They propose automating the selection of DR projections (VoyagerDR) as a pragmatic step, while emphasizing the need to maintain user agency and explainability. The work highlights the practical significance of improving how DR techniques are chosen and evaluated, aiming to enhance the reliability of visual analytics and stimulate broader discussion on responsible ML usage.

Abstract

Misuses of t-SNE and UMAP in visual analytics have become increasingly common. For example, although t-SNE and UMAP projections often do not faithfully reflect the original distances between clusters, practitioners frequently use them to investigate inter-cluster relationships. We investigate why this misuse occurs, and discuss methods to prevent it. To that end, we first review 136 papers to verify the prevalence of the misuse. We then interview researchers who have used dimensionality reduction (DR) to understand why such misuse occurs. Finally, we interview DR experts to examine why previous efforts failed to address the misuse. We find that the misuse of t-SNE and UMAP stems primarily from limited DR literacy among practitioners, and that existing attempts to address this issue have been ineffective. Based on these insights, we discuss potential paths forward, including the controversial but pragmatic option of automating the selection of optimal DR projections to prevent misleading analyses.

Figures (6)

• Figure 1: Comparison of t-SNE, UMAP, densMAP narayan21nature, and UMATO jeon22vis projections of a 2D dataset. Although t-SNE and UMAP do not faithfully represent CLUSTER NONCLICKclusternotclick1 id3, clusternotclick, clusterclickeddistnotclickid2, id4cluster densityCLUSTER CLICKclusterclicked0 id3, clusterclicked, clusternotclickdistnotclickid2, id4, distclickedcluster densityor DIST NONCLICKdistnotclick1 id2, distnotclick, distclickedclusternotclickid3, id4, clusterclickeddistances between data points,DIST CLICKdistclicked0 id2, distclicked, distnotclickclusternotclickid3, id4, clusterclickeddistances between data points,they are often misused to analyze such structures. Our research investigates why this misuse happens and explores strategies to address it. [This figure is interactive in Adobe Acrobat reader, where the underlined texts can be clicked]
• Figure 2: Illustrations of the analytic tasks using DR and their alignment to local and global DR techniques. Our literature review identifies seven types of analytic tasks using DR. t-SNE and UMAP are suitable for neighborhood identification, outlier identification, and cluster identification tasks but inappropriate for other tasks.
• Figure 3: The trend of accumulated number of papers that use (a) or misuse (b) of four major DR techniques. We collect papers published from 2008, the year t-SNE is introduced. Note that UMAP's data also starts from the year it is released (2018). [This figure is interactive in Adobe Acrobat reader, where the underlined texts can be clicked]
• Figure 4: The ratio of appropriate use and misuse of DR techniques by analytic tasks. DR is appropriately used for tasks that align with local techniques (top 3) but not for those that align with global techniques (bottom 4). This result indicates that local techniques (e.g., t-SNE, UMAP) are overtrusted even for tasks that are not suitable. Papers are marked as "fully misused" if all tasks targeted by the paper are not supported by the employed DR technique. Papers with partial support are marked as "partially misused."
• Figure 5: The number of appropriate use and misuse of DR by techniques (left) and their ratio (right). As with \ref{['fig:error-tasks']}, papers are marked as "fully misused" and "partially misused" if all tasks targeted by the paper are entirely or partially not supported by the used DR techniques. The analysis reveals that t-SNE and UMAP dominate the misuse of DR.