IPBC: An Interactive Projection-Based Framework for Human-in-the-Loop Semi-Supervised Clustering of High-Dimensional Data
Mohammad Zare
TL;DR
IPBC tackles the challenge of clustering high-dimensional data by enabling humans to steer a nonlinear projection learning process through must-link and cannot-link constraints. By iteratively refining a 2D embedding and then clustering in this space, IPBC achieves higher external and internal clustering metrics while providing interpretable explanations based on original features. The approach demonstrates that interactive, explainable collaboration between user intuition and machine learning can yield more accurate and trustworthy clustering results, with practical workflow benefits illustrated on image and single-cell datasets. Overall, IPBC reframes clustering as a collaborative discovery process that leverages visualization, constraint-based guidance, and lightweight explainability to improve both performance and interpretability.
Abstract
High-dimensional datasets are increasingly common across scientific and industrial domains, yet they remain difficult to cluster effectively due to the diminishing usefulness of distance metrics and the tendency of clusters to collapse or overlap when projected into lower dimensions. Traditional dimensionality reduction techniques generate static 2D or 3D embeddings that provide limited interpretability and do not offer a mechanism to leverage the analyst's intuition during exploration. To address this gap, we propose Interactive Project-Based Clustering (IPBC), a framework that reframes clustering as an iterative human-guided visual analysis process. IPBC integrates a nonlinear projection module with a feedback loop that allows users to modify the embedding by adjusting viewing angles and supplying simple constraints such as must-link or cannot-link relationships. These constraints reshape the objective of the projection model, gradually pulling semantically related points closer together and pushing unrelated points further apart. As the projection becomes more structured and expressive through user interaction, a conventional clustering algorithm operating on the optimized 2D layout can more reliably identify distinct groups. An additional explainability component then maps each discovered cluster back to the original feature space, producing interpretable rules or feature rankings that highlight what distinguishes each cluster. Experiments on various benchmark datasets show that only a small number of interactive refinement steps can substantially improve cluster quality. Overall, IPBC turns clustering into a collaborative discovery process in which machine representation and human insight reinforce one another.