Quantifying Visual Properties of GAM Shape Plots: Impact on Perceived Cognitive Load and Interpretability

TL;DR

This study addresses how the visual properties of GAM shape plots influence cognitive load and interpretability. It develops five final visual-property metrics (Graph Length, Polynomial Degree, Visual Chunks, Number of Kinks, Avg Kink Distance) extracted via Python, and validates them through a user study with 57 participants across 144 plots, using linear regression and log-transformations with validation via rankings and binary choices. The number of kinks emerges as the strongest predictor, explaining 86.4% of the variance in perceived cognitive load, and a simple predictive model is proposed: $PCL = 1.724 + 1.377 \times \ln(1 + \text{number of kinks})$. The work provides Python tools and a public dataset to enable rapid, context-independent assessment of GAM shape-plot interpretability during model development, reducing reliance on costly user studies.

Abstract

Generalized Additive Models (GAMs) offer a balance between performance and interpretability in machine learning. The interpretability aspect of GAMs is expressed through shape plots, representing the model's decision-making process. However, the visual properties of these plots, e.g. number of kinks (number of local maxima and minima), can impact their complexity and the cognitive load imposed on the viewer, compromising interpretability. Our study, including 57 participants, investigates the relationship between the visual properties of GAM shape plots and cognitive load they induce. We quantify various visual properties of shape plots and evaluate their alignment with participants' perceived cognitive load, based on 144 plots. Our results indicate that the number of kinks metric is the most effective, explaining 86.4% of the variance in users' ratings. We develop a simple model based on number of kinks that provides a practical tool for predicting cognitive load, enabling the assessment of one aspect of GAM interpretability without direct user involvement.

Figures (3)

• Figure 1: Visual properties of shape plots for the same feature vary based on underlying model architecture, thereby affecting both users' perceived cognitive load and predicted cognitive load.
• Figure 2: Visual representation of the metrics for three different shape plots. The columns indicate three different types of GAMs. The rows represent the five metrics selected in several workshops.
• Figure 3: Visualization of the logarithmic transformation's effect on the metric-based model for the number of kinks metric.