Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SHIFT: An Interdisciplinary Framework for Scaffolding Human Attention and Understanding in Explanatory Tasks

André Groß, Birte Richter, Britta Wrede

TL;DR

SHIFT presents an interdisciplinary, domain-independent framework for adaptive scaffolding in explanatory human-robot interactions. It encodes human cognitive states into a reduced set of observations and uses a pre-configured interdisciplinary scoring system to guide $Q$-learning-based scaffolding decisions, enabling rapid adaptation with limited interactions. The approach integrates monitoring, decision making, and scaffolding generation within a ROS-enabled, dockerized architecture and demonstrates improved learning efficiency across diverse user types in simulation. The work lays groundwork for real-world HRI studies by refining the scoring system, exploring correlations between cognition and verbal scaffolding, and enabling runtime human-in-the-loop configuration. The practical impact is a configurable, explainable scaffolding framework that can speed up user learning and task performance in robotic explanations.

Abstract

In this work, we present a domain-independent approach for adaptive scaffolding in robotic explanation generation to guide tasks in human-robot interaction. We present a method for incorporating interdisciplinary research results into a computational model as a pre-configured scoring system implemented in a framework called SHIFT. This involves outlining a procedure for integrating concepts from disciplines outside traditional computer science into a robotics computational framework. Our approach allows us to model the human cognitive state into six observable states within the human partner model. To study the pre-configuration of the system, we implement a reinforcement learning approach on top of our model. This approach allows adaptation to individuals who deviate from the configuration of the scoring system. Therefore, in our proof-of-concept evaluation, the model's adaptability on four different user types shows that the models' adaptation performs better, i.e., recouped faster after exploration and has a higher accumulated reward with our pre-configured scoring system than without it. We discuss further strategies of speeding up the learning phase to enable a realistic adaptation behavior to real users. The system is accessible through docker and supports querying via ROS.

SHIFT: An Interdisciplinary Framework for Scaffolding Human Attention and Understanding in Explanatory Tasks

TL;DR

SHIFT presents an interdisciplinary, domain-independent framework for adaptive scaffolding in explanatory human-robot interactions. It encodes human cognitive states into a reduced set of observations and uses a pre-configured interdisciplinary scoring system to guide -learning-based scaffolding decisions, enabling rapid adaptation with limited interactions. The approach integrates monitoring, decision making, and scaffolding generation within a ROS-enabled, dockerized architecture and demonstrates improved learning efficiency across diverse user types in simulation. The work lays groundwork for real-world HRI studies by refining the scoring system, exploring correlations between cognition and verbal scaffolding, and enabling runtime human-in-the-loop configuration. The practical impact is a configurable, explainable scaffolding framework that can speed up user learning and task performance in robotic explanations.

Abstract

In this work, we present a domain-independent approach for adaptive scaffolding in robotic explanation generation to guide tasks in human-robot interaction. We present a method for incorporating interdisciplinary research results into a computational model as a pre-configured scoring system implemented in a framework called SHIFT. This involves outlining a procedure for integrating concepts from disciplines outside traditional computer science into a robotics computational framework. Our approach allows us to model the human cognitive state into six observable states within the human partner model. To study the pre-configuration of the system, we implement a reinforcement learning approach on top of our model. This approach allows adaptation to individuals who deviate from the configuration of the scoring system. Therefore, in our proof-of-concept evaluation, the model's adaptability on four different user types shows that the models' adaptation performs better, i.e., recouped faster after exploration and has a higher accumulated reward with our pre-configured scoring system than without it. We discuss further strategies of speeding up the learning phase to enable a realistic adaptation behavior to real users. The system is accessible through docker and supports querying via ROS.

Paper Structure

This paper contains 22 sections, 8 equations, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Decision-Making in Social Robotics Tutoring
  4. Adaptive Scaffolding in Explanation Generation
  5. Research Scope
  6. Computational Model for Adaptive Scaffolding when Explaining a Task
  7. State-Space: Observations for Monitoring
  8. Processing Capacity
  9. Visual Distribution of Attention
  10. Task Awareness
  11. Actions: Explanation Scaffolding Strategies
  12. State Reduction: Cognitive States of the Partner Model
  13. Decision-Making: Adaptive Scaffolding
  14. Reward: User's Task-Performance
  15. Integration in Real-World Application
  16. ...and 7 more sections

Figures (3)

  • Figure 1: Interaction between human and robot in a social context. The robot using individual scaffolding strategies to guide during task solving.
  • Figure 2: The computational model of SHIFT with information flow. From the monitoring of the partner model of the human through the decision-making system to the generation of scaffolding actions for the hri.
  • Figure 3: Accumulated reward of Q-learning algorithm in 100 simulated interactions with and without the initialization of a Q-table based on the scoring system each. (A) is the optimal user, where user performance corresponds to (B) 1-dimensional (processing capacity) (C) 2-dimensional (processing capacity and task awareness) (D) 3-dimensional (processing capacity, task awareness and gaze distribution) error in the pre-configuration for a negation strategy.