UX Challenges in Implementing an Interactive B2B Customer Segmentation Tool

TL;DR

This paper addresses UX challenges in interpreting unsupervised clustering for B2B customer segmentation by presenting an Interactive Machine Learning (IML) prototype evaluated with domain experts using a real-world, multi-year sales dataset. The authors detail a two-phase study: an initial technology-probe-driven elicitation of requirements and a subsequent piloting phase that adds semantic labeling, explanations, and runtime customization, enabling domain users to actively label, adjust, and compare segmentation outputs. Key findings show that user agency, semantic mapping between labels and features, and explanation mechanisms significantly improve sense-making and trust in unsupervised segmentation, though human-in-the-loop interaction remains necessary to adapt models to dynamic business contexts. The work highlights practical design patterns for IML in B2B settings, including interactive clustering, feature-focused semantic labeling, and explanation-driven decision support, with implications for broader adoption in domains requiring subjective expertise and contextual adaptation.

Abstract

In our effort to implement an interactive customer segmentation tool for a global manufacturing company, we identified user experience (UX) challenges with technical implications. The main challenge relates to domain users' effort, in our case sales experts, to interpret the clusters produced by an unsupervised Machine Learning (ML) algorithm, for creating a customer segmentation. An additional challenge is what sort of interactions should such a tool support to enable meaningful interpretations of the output of clustering models. In this case study, we describe what we learned from implementing an Interactive Machine Learning (IML) prototype to address such UX challenges. We leverage a multi-year real-world dataset and domain experts' feedback from a global manufacturing company to evaluate our tool. We report what we found to be effective and wish to inform designers of IML systems in the context of customer segmentation and other related unsupervised ML tools.

Figures (3)

• Figure 1: Interactive cluster analysis allowed users to A) visualize, interact, and filter with the graph (and its elements) to make sense of data by tweaking visualization elements, B) select models (by changing cluster counts) to contextualize output, C) Modify or re-label segments to make connections and contextual understanding, D) visualize and an overview of descriptive statistics about each cluster in the selected model to overall understanding. For confidentiality reasons, the visualization is modified with anonymous data and labels.
• Figure 2: Users can type and map their labels with the model’s features -in this case, profit and volume of products bought. For instance, in this case, the user specifies a high profit and volume of products bought for cluster 0 (i.e., one can imagine this cluster grouping really good customers) and a moderate profit and volume of products bought for cluster 1 (i.e., one can imagine this cluster grouping customers who might have the potential to grow in terms of the profit and volume of products that they buy.)
• Figure 3: Inspection allows users to modify and adapt segments and use features to support their decision-making. Users can A) interact with and adjust the visualization to inspect the segmentation from different angles B) visualize several graphs and modify visualization by selecting the features they want to display C) modify the selected instance from the visualization D) visualize the historical information of the selected instance E) view various explanation visualizations to inspect models F) Relabel a cluster. For instance, a user is visualizing "CustomerX" by adjusting visualization to 3D and selecting the features: profit, average profit per ton, and volume for each axis. For confidentiality reasons, the visualization is modified with anonymous data and labels.