Cause-effect perception in an object place task

TL;DR

This study investigates how people perceive and learn causal structure in a realistic sensorimotor task implemented in VR with haptic weight cues. It distinguishes learning at conscious cognitive vs. sensorimotor levels and compares human performance to structure-learning algorithms (FCI, PC, FGES) applied to the same data. Findings show partial learning: subjects reliably identify weight–breakability but struggle with directionality and color–breakability; sensorimotor representations reveal weight–force coupling that strengthens over sessions while color–force remains noisy. Causal-discovery algorithms recover the ground-truth structure across sessions, suggesting partial alignment between human and algorithmic causal representations and revealing potential dissociations between levels. The work highlights how sensorimotor learning can inform causal representation research and the development of human-aware AI systems.

Abstract

We conducted an exploratory study in virtual reality to examine if people can discover causal relations in a realistic sensorimotor context and how such learning is represented at different processing levels (conscious-cognitive vs. sensorimotor). Additionally, we explored the relation between human causal learning and state-of-the-art causal discovery algorithms. The task consisted of placing a glass on a surface, that breaks if the contact force exeeded its breakability threshold, determined by weight and color. Ecological validity was enhanced by haptic rendering simulating weight and contact forces. Participants were asked to repeatedly transport and place glasses of varying weights and colors on a surface without breaking them. For success, participants had to discover the underlying causal structure. The trials were conducted over three sessions, reflecting naive, exploratory, consolidated and causally aware behavior, with questionnaires assessing conscious causal understanding of the task's causal structure. Sensorimotor representations were inferred by applying causal-discovery algorithms (PC, FCI, FGES) to the recorded trial-by-trial variables, and conditional mutual information was used to quantify the strength of causal influence on the sensorimotor level. Results show that (i) participants identified the weight-breakability link (76% correct after experiment) and the color-breakability link (43%) but struggle to infer causal direction. (ii) Sensorimotor analysis revealed a robust weight-force coupling increasing across sessions, whereas for color-force it was weak and noisy, yet mutual information indicated an attempted learning. (iii) Discovery algorithms recovered the causal structure across sessions. Together, these findings indicate that humans can, partially, perceive the causal structure of the task, with partially dissociated conscious and sensorimotor representations.

Figures (20)

• Figure 1: The ground truth causal structure of the experimental setup. Upper left sub-graph: causal dependencies between glass properties. Lower subgraph: causal determinants of possible glass destruction.
• Figure 2: Causal graphs relevant for the sensorimotor causal representation. Left: ground truth causal structure of the experimental setup. Center: Internal representation enforced to be used for sensorimotor control. Right: The corresponding simplified proxy graph of the sensorimotor causal representation.
• Figure 3: Full causal graph with ground truth causal relations of experimental setup and of the (hypothetical) sensorimotor representation.
• Figure 4: Experiment setup
• Figure 5: Aggregated results of the questionnaire. The thickness of the lines corresponds to the fraction of how many participants identified the relation between the connected variables. This fraction is also shown on the given lines. The dashed connection indicates that this connection is erroneous.