Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Intuitive Programming, Adaptive Task Planning, and Dynamic Role Allocation in Human-Robot Collaboration

Marta Lagomarsino, Elena Merlo, Andrea Pupa, Timo Birr, Franziska Krebs, Cristian Secchi, Tamim Asfour, Arash Ajoudani

TL;DR

This paper reframes human-robot collaboration (HRC) through an information-theoretic lens, treating the interaction as a bidirectional communication channel where humans convey intent via multi-modal signals and robots respond with interpretable feedback. It surveys intuitive programming approaches, adaptive task planning architectures (including PDDL, HTN, AND/OR graphs, BTs, and foundation models), and dynamic role allocation strategies driven by human load, task complexity, and context. The work highlights the role of learning from demonstrations, large-scale data, and multi-modal inputs to build robust task models and adaptable plans, while addressing safety and human factors via a control abstraction layer and intuitive feedback mechanisms. Together, these elements aim to enable online co-adaptation, personalized partner models, and proactive assistance, driving more natural, trustworthy, and scalable HRC in real-world settings.

Abstract

Remarkable capabilities have been achieved by robotics and AI, mastering complex tasks and environments. Yet, humans often remain passive observers, fascinated but uncertain how to engage. Robots, in turn, cannot reach their full potential in human-populated environments without effectively modeling human states and intentions and adapting their behavior. To achieve a synergistic human-robot collaboration (HRC), a continuous information flow should be established: humans must intuitively communicate instructions, share expertise, and express needs. In parallel, robots must clearly convey their internal state and forthcoming actions to keep users informed, comfortable, and in control. This review identifies and connects key components enabling intuitive information exchange and skill transfer between humans and robots. We examine the full interaction pipeline: from the human-to-robot communication bridge translating multimodal inputs into robot-understandable representations, through adaptive planning and role allocation, to the control layer and feedback mechanisms to close the loop. Finally, we highlight trends and promising directions toward more adaptive, accessible HRC.

Intuitive Programming, Adaptive Task Planning, and Dynamic Role Allocation in Human-Robot Collaboration

TL;DR

This paper reframes human-robot collaboration (HRC) through an information-theoretic lens, treating the interaction as a bidirectional communication channel where humans convey intent via multi-modal signals and robots respond with interpretable feedback. It surveys intuitive programming approaches, adaptive task planning architectures (including PDDL, HTN, AND/OR graphs, BTs, and foundation models), and dynamic role allocation strategies driven by human load, task complexity, and context. The work highlights the role of learning from demonstrations, large-scale data, and multi-modal inputs to build robust task models and adaptable plans, while addressing safety and human factors via a control abstraction layer and intuitive feedback mechanisms. Together, these elements aim to enable online co-adaptation, personalized partner models, and proactive assistance, driving more natural, trustworthy, and scalable HRC in real-world settings.

Abstract

Remarkable capabilities have been achieved by robotics and AI, mastering complex tasks and environments. Yet, humans often remain passive observers, fascinated but uncertain how to engage. Robots, in turn, cannot reach their full potential in human-populated environments without effectively modeling human states and intentions and adapting their behavior. To achieve a synergistic human-robot collaboration (HRC), a continuous information flow should be established: humans must intuitively communicate instructions, share expertise, and express needs. In parallel, robots must clearly convey their internal state and forthcoming actions to keep users informed, comfortable, and in control. This review identifies and connects key components enabling intuitive information exchange and skill transfer between humans and robots. We examine the full interaction pipeline: from the human-to-robot communication bridge translating multimodal inputs into robot-understandable representations, through adaptive planning and role allocation, to the control layer and feedback mechanisms to close the loop. Finally, we highlight trends and promising directions toward more adaptive, accessible HRC.

Paper Structure

This paper contains 28 sections, 3 figures.

Table of Contents

  1. INTRODUCTION
  2. Relation to other surveys
  3. HUMAN TO ROBOT INTUITIVE BRIDGE
  4. Multi-modal Input for Intuitive Programming
  5. Natural Language
  6. Gesture and Body Language Based Communication
  7. Physiological Signals
  8. Eye Tracking
  9. How can humans intuitively transfer skills to robots?
  10. Learning Task Models from Human Demonstrations
  11. Learning Constraints from Human Demonstration Data
  12. Policy Learning from Visual Observations
  13. Learning from Large-Scale Data
  14. Unstructured Internet-Scale Data
  15. Structured and Semi-Structured Motion Databases
  16. ...and 13 more sections

Figures (3)

  • Figure 1: Synergistic HRC systems function as dynamic, bidirectional channels where humans and robots continuously exchange information to enable shared understanding and coordinated action.
  • Figure 2: The "intuitive bridge" establishes a bidirectional interface between the human-interpretable layer and the robot-interpretable layers. It functions by translating high-dimensional, multi-modal human inputs into structured symbolic representations and sensorimotor signals that robots can process to construct and adapt task models (see Section 2). In the opposite direction, this bridge enables the transformation of robot-generated outputs into human-interpretable feedback channels, fostering mutual understanding between the human and the robot (see Section 6).
  • Figure 3: Left: Distribution of papers in the literature addressing dynamic role allocation, categorized by the strategy employed (rule-based, optimization, or learning) and their associated task planning methods. Right: Overview of the papers included in this review, grouped by the dynamic role allocation strategy and the cost guiding the allocation.