Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Towards Metrics for Evaluating Creativity in Visualisation Design

Aron E Owen, Jonathan C Roberts

TL;DR

The paper addresses the lack of metrics for evaluating creativity in visualization sketches, focusing on low-fidelity designs. It introduces the Rowen Test, a four-dimension scoring framework (Quantity, Correctness, Novelty, Feasibility) administered on a 0 to 25 per-dimension scale, demonstrated with eight visualization experts producing 64 sketches. The study reports positive feedback on ease of use and the test's potential to foster rapid ideation, while noting the need for clearer scoring guidance and more contextual data. The work provides a practical, scalable foundation for objective creativity assessment in visualization design and suggests directions for refinement and broader adoption.

Abstract

Creativity in visualisation design is essential for designers and data scientists who need to present data in innovative ways. It is often achieved through sketching or drafting low-fidelity prototypes. However, judging this innovation is often difficult. A creative visualisation test would offer a structured approach to enhancing visual thinking and design skills, which are vital across many fields. Such a test can facilitate objective evaluation, skill identification, benchmarking, fostering innovation, and improving learning outcomes. In developing such a test, we propose focusing on four criteria: Quantity, Correctness, Novelty, and Feasibility. These criteria integrate into a test that is easy to administer. We name it the Rowen Test of Creativity in Visualisation Design; We introduce the test, scoring system and results from using eight visualisation experts.

Towards Metrics for Evaluating Creativity in Visualisation Design

TL;DR

The paper addresses the lack of metrics for evaluating creativity in visualization sketches, focusing on low-fidelity designs. It introduces the Rowen Test, a four-dimension scoring framework (Quantity, Correctness, Novelty, Feasibility) administered on a 0 to 25 per-dimension scale, demonstrated with eight visualization experts producing 64 sketches. The study reports positive feedback on ease of use and the test's potential to foster rapid ideation, while noting the need for clearer scoring guidance and more contextual data. The work provides a practical, scalable foundation for objective creativity assessment in visualization design and suggests directions for refinement and broader adoption.

Abstract

Creativity in visualisation design is essential for designers and data scientists who need to present data in innovative ways. It is often achieved through sketching or drafting low-fidelity prototypes. However, judging this innovation is often difficult. A creative visualisation test would offer a structured approach to enhancing visual thinking and design skills, which are vital across many fields. Such a test can facilitate objective evaluation, skill identification, benchmarking, fostering innovation, and improving learning outcomes. In developing such a test, we propose focusing on four criteria: Quantity, Correctness, Novelty, and Feasibility. These criteria integrate into a test that is easy to administer. We name it the Rowen Test of Creativity in Visualisation Design; We introduce the test, scoring system and results from using eight visualisation experts.
Paper Structure (4 sections, 1 figure)

This paper contains 4 sections, 1 figure.

Table of Contents

  1. Introduction
  2. Related Work
  3. Building the metrics and evaluation
  4. Discussion and conclusion

Figures (1)

  • Figure 1: The Rowen test (top left) shows the eight participants. They self-assessed their work. Quantity is easy to judge, with P8 being the most and P5 being the least. Creativity is highest in P1, with P7 focusing on typical charts (including line graphs, bar charts, stacked bars and so forth).