Establishing Heuristics for Improving the Usability of GUI Machine Learning Tools for Novice Users

TL;DR

This work targets the usability gap in GUI-based ML tools for novices by extending Nielsen's heuristics with a data-driven, four-phase process. Through heuristic evaluation, cognitive walkthroughs, user surveys, and SUS testing on Weka and KNIME, the authors derive 14 domain-specific heuristics and validate them by building a prototype that adheres to these guidelines. The prototype—with features like a guiding octopus, a task log, and an explainability tab—achieves higher usability, including 100% task success and SUS of 74.62, compared with the baseline tools. The study demonstrates that tailored heuristics can significantly improve novice user interaction with GUI ML tools, and it outlines directions for longitudinal validation and growth-adaptive interfaces.

Abstract

Machine learning (ML) tools with graphical user interfaces (GUI) are facing demand from novice users who do not have the background of their underlying concepts. These tools are frequently complex and pose unique challenges in terms of interaction and comprehension by novice users. There is yet to be an established set of usability heuristics to guide and assess GUI ML tool design. To address this gap, in this paper, we extend Nielsen's heuristics for evaluating GUI ML Tools through a set of empirical evaluations. To validate the proposed heuristics, user testing was conducted by novice users on a prototype that reflects those heuristics. Based on the results of the evaluations, our new heuristics set improves upon existing heuristics in the context of ML tools. It can serve as a resource for practitioners designing and evaluating these tools.

Figures (19)

• Figure 1: Research Methodology
• Figure 2: The nine stages of the machine learning workflow amershi2019software.
• Figure 3: Task Analysis
• Figure 4: Issues discovered during the Heuristic Evaluation heuristic number.
• Figure 5: Success Rate Ratio