Learning Human Detected Differences in Directed Acyclic Graphs

TL;DR

This work tackles the divergence between human perception and formal graph similarity in visual comparisons of directed acyclic graphs (DAGs). It proposes learning human-detected structural differences by combining a knowledge-based data augmentation method (Difference Factors Simulation, DFS) with a convolutional neural network (Mask R-CNN) designed to operate on image pairs of DAG visualizations. The dataset comprises tree-like and sparse DAGs with 10,000 pairs, annotated to reflect human-detected differences, enabling supervised learning of which graph elements humans would deem different. Empirical results show high recall (~0.94) and competitive AP (~0.86) when predicting human-detected differences, demonstrating that the model can highlight differences that align with human perception and potentially improve the credibility and effectiveness of algorithm-assisted visual comparison systems. The work also discusses image-based versus graph-based learning approaches and outlines future directions for adaptive thresholds and user-specific models to further align machine predictions with human notions of structural differences in graphs.

Abstract

Prior research has shown that human perception of similarity differs from mathematical measures in visual comparison tasks, including those involving directed acyclic graphs. This divergence can lead to missed differences and skepticism about algorithmic results. To address this, we aim to learn the structural differences humans detect in graphs visually. We want to visualize these human-detected differences alongside actual changes, enhancing credibility and aiding users in spotting overlooked differences. Our approach aligns with recent research in machine learning capturing human behavior. We provide a data augmentation algorithm, a dataset, and a machine learning model to support this task. This work fills a gap in learning differences in directed acyclic graphs and contributes to better comparative visualizations.

Figures (27)

• Figure 1: Visual analytics pipeline, adapted from Keim et al. Keim2008, to illustrate the benefit of our convolutional neural network, which is able to predict human detected differences in directed acyclic graphs, in a visual interactive system supporting comparisons.
• Figure 2: A set of two faces. Determining the set of distinctive features of $a$ compared to $b$ based on the notion of a set-theoretical matching function of Tversky's pairwise similarity model this results in the smiling mouth. The Figure is an excerpt of Tversky's Figure in tversky1977features.
• Figure 3: The visual comparison workflow according to von Landesberger TatianaHabil (Figure based on original Figure from TatianaHabil).
• Figure 4: Dynamic graph graph with structural changes -- additions and deletions of nodes -- from Giacomo2015 (Figure based on original Figure from Giacomo2015).
• Figure 5: Difference Factors Simulation (DFS) algorithm -- schematic representation. (Figure based on original Figure from AlenaMA)