Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Decoupling Judgment and Decision Making: A Tale of Two Tails

Başak Oral, Pierre Dragicevic, Alexandru Telea, Evanthia Dimara

Abstract

Is it true that if citizens understand hurricane probabilities, they will make more rational decisions for evacuation? Finding answers to such questions is not straightforward in the literature because the terms judgment and decision making are often used interchangeably. This terminology conflation leads to a lack of clarity on whether people make suboptimal decisions because of inaccurate judgments of information conveyed in visualizations or because they use alternative yet currently unknown heuristics. To decouple judgment from decision making, we review relevant concepts from the literature and present two preregistered experiments (N=601) to investigate if the task (judgment vs. decision making), the scenario (sports vs. humanitarian), and the visualization (quantile dotplots, density plots, probability bars) affect accuracy. While experiment 1 was inconclusive, we found evidence for a difference in experiment 2. Contrary to our expectations and previous research, which found decisions less accurate than their direct-equivalent judgments, our results pointed in the opposite direction. Our findings further revealed that decisions were less vulnerable to status-quo bias, suggesting decision makers may disfavor responses associated with inaction. We also found that both scenario and visualization types can influence peoples judgments and decisions. Although effect sizes are not large and results should be interpreted carefully, we conclude that judgments cannot be safely used as proxy tasks for decision making, and discuss implications for visualization research and beyond.

Decoupling Judgment and Decision Making: A Tale of Two Tails

Abstract

Is it true that if citizens understand hurricane probabilities, they will make more rational decisions for evacuation? Finding answers to such questions is not straightforward in the literature because the terms judgment and decision making are often used interchangeably. This terminology conflation leads to a lack of clarity on whether people make suboptimal decisions because of inaccurate judgments of information conveyed in visualizations or because they use alternative yet currently unknown heuristics. To decouple judgment from decision making, we review relevant concepts from the literature and present two preregistered experiments (N=601) to investigate if the task (judgment vs. decision making), the scenario (sports vs. humanitarian), and the visualization (quantile dotplots, density plots, probability bars) affect accuracy. While experiment 1 was inconclusive, we found evidence for a difference in experiment 2. Contrary to our expectations and previous research, which found decisions less accurate than their direct-equivalent judgments, our results pointed in the opposite direction. Our findings further revealed that decisions were less vulnerable to status-quo bias, suggesting decision makers may disfavor responses associated with inaction. We also found that both scenario and visualization types can influence peoples judgments and decisions. Although effect sizes are not large and results should be interpreted carefully, we conclude that judgments cannot be safely used as proxy tasks for decision making, and discuss implications for visualization research and beyond.
Paper Structure (43 sections, 1 equation, 4 figures)

This paper contains 43 sections, 1 equation, 4 figures.

Table of Contents

  1. Introduction
  2. Related Work
  3. Judgment in Visualization
  4. Decision Making in Visualization
  5. Conflating Judgment and Decision Making
  6. Judgment and Decision Making in Psychology
  7. Judgment & Decision Making Definitions
  8. Judgment & Decision Making Studies
  9. Research Hypotheses
  10. Design Rationale: Judgment Vs. Decision
  11. Differences Between Tasks
  12. Identical Design Choices Between Tasks
  13. Experiment 1
  14. Visualizations
  15. Scenarios and Stimuli
  16. ...and 28 more sections

Figures (4)

  • Figure 1: The three visualizations used in both experiments: quantile dotplot (left), density plot (middle), and probability bar (right).
  • Figure 2: Example of the decision task condition with the sports scenario and quantile dotplot visualization. The judgment task version was identical, except for modifications in the title and question, highlighted in blue.
  • Figure 3: Results for H${_1}$ - H$_5$ showing CI of average accuracy and bias percentage per condition for experiments 1 (left) and 2 (right). Ellipses depict the difference (black CI) between average judgment (blue CI) and decision (red CI) accuracies (for H${_1}$ - H${_4}$) and bias percentage (H${_5}$). Cohen's d values for each difference are bottom-right of the ellipses. Chart titles (e.g., total, sports) indicate the specific condition.
  • Figure 4: Accuracy scores ($x$ axis) vs. participant count ($y$ axis) colored by response time for both experiments. Log-transformed time ranges: fast (0.52-1.43], medium (1.43-1.74], slow (1.74-3.04], corresponding to minutes: (1.68-4.18], (4.18-5.70], (5.70-20.90].