Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Human-Centered Approach for Bootstrapping Causal Graph Creation

Minh Q. Tram, Nolan B. Gutierrez, William J. Beksi

TL;DR

This work presents initial results towards a human-centered augmented reality framework for creating causal graphical models, which bootstraps the causal discovery process by involving humans in selecting variables, establishing relationships, performing interventions, generating counterfactual explanations, and evaluating the resulting causal graph at every step.

Abstract

Causal inference, a cornerstone in disciplines such as economics, genomics, and medicine, is increasingly being recognized as fundamental to advancing the field of robotics. In particular, the ability to reason about cause and effect from observational data is crucial for robust generalization in robotic systems. However, the construction of a causal graphical model, a mechanism for representing causal relations, presents an immense challenge. Currently, a nuanced grasp of causal inference, coupled with an understanding of causal relationships, must be manually programmed into a causal graphical model. To address this difficulty, we present initial results towards a human-centered augmented reality framework for creating causal graphical models. Concretely, our system bootstraps the causal discovery process by involving humans in selecting variables, establishing relationships, performing interventions, generating counterfactual explanations, and evaluating the resulting causal graph at every step. We highlight the potential of our framework via a physical robot manipulator on a pick-and-place task.

A Human-Centered Approach for Bootstrapping Causal Graph Creation

TL;DR

This work presents initial results towards a human-centered augmented reality framework for creating causal graphical models, which bootstraps the causal discovery process by involving humans in selecting variables, establishing relationships, performing interventions, generating counterfactual explanations, and evaluating the resulting causal graph at every step.

Abstract

Causal inference, a cornerstone in disciplines such as economics, genomics, and medicine, is increasingly being recognized as fundamental to advancing the field of robotics. In particular, the ability to reason about cause and effect from observational data is crucial for robust generalization in robotic systems. However, the construction of a causal graphical model, a mechanism for representing causal relations, presents an immense challenge. Currently, a nuanced grasp of causal inference, coupled with an understanding of causal relationships, must be manually programmed into a causal graphical model. To address this difficulty, we present initial results towards a human-centered augmented reality framework for creating causal graphical models. Concretely, our system bootstraps the causal discovery process by involving humans in selecting variables, establishing relationships, performing interventions, generating counterfactual explanations, and evaluating the resulting causal graph at every step. We highlight the potential of our framework via a physical robot manipulator on a pick-and-place task.
Paper Structure (14 sections, 2 equations, 5 figures, 1 algorithm)

This paper contains 14 sections, 2 equations, 5 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background and Motivation
  3. Causal Graphs and Causal Inference
  4. Motivation
  5. Human-Centered Causal Graphical Models
  6. Component Overview and Consideration
  7. Interaction and Visualization Modalities
  8. Causal Graph Construction and Intervention
  9. Physical Robot Actuation and Simulation
  10. Evaluation
  11. Causal Graph Creation
  12. Causal Graph Intervention
  13. Discussion
  14. Conclusion

Figures (5)

  • Figure 1: The point of view of the operator from the AR overlaid physical workspace. The operator can interact with the robot and provide additional context to aid the robot's understanding via on-the-fly construction of causal graphs.
  • Figure 2: A causal graph representing the relationship between a robot's battery level $B$, the terrain roughness $T$, and the robot's velocity $V$ during navigation. The solid edge represents a well-established cause and effect, while the dashed edge represents an indirect or latent confounding variable.
  • Figure 3: The visualization and interaction pipeline of our proposed framework. The operator can interact with the robot using an overlay interface via various modes and provide context hinting directly onto the workspace to aid the robot in its understanding of the scene.
  • Figure 4: An example of the relationships between the operator-identified variables for a pick-and-place task. Each edge corresponds to a direct causal relationship between the connected variables. Changes in the source of an edge have a direct consequence on the target of an edge.
  • Figure 5: The operator interacting with the robot and environment through the AR interface to construct a causal graph. Nodes and edges are visualized and can be manipulated directly in the AR space.