Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Multi-Modal MPPI and Active Inference for Reactive Task and Motion Planning

Yuezhe Zhang, Corrado Pezzato, Elia Trevisan, Chadi Salmi, Carlos Hernández Corbato, Javier Alonso-Mora

TL;DR

This work proposes a method for reactive TAMP to cope with runtime uncertainties and disturbances, which combines an Active Inference planner for adaptive high-level action selection and a novel Multi-Modal Model Predictive Path Integral controller (M3P2I) for low-level control.

Abstract

Task and Motion Planning (TAMP) has made strides in complex manipulation tasks, yet the execution robustness of the planned solutions remains overlooked. In this work, we propose a method for reactive TAMP to cope with runtime uncertainties and disturbances. We combine an Active Inference planner (AIP) for adaptive high-level action selection and a novel Multi-Modal Model Predictive Path Integral controller (M3P2I) for low-level control. This results in a scheme that simultaneously adapts both high-level actions and low-level motions. The AIP generates alternative symbolic plans, each linked to a cost function for M3P2I. The latter employs a physics simulator for diverse trajectory rollouts, deriving optimal control by weighing the different samples according to their cost. This idea enables blending different robot skills for fluid and reactive plan execution, accommodating plan adjustments at both the high and low levels to cope, for instance, with dynamic obstacles or disturbances that invalidate the current plan. We have tested our approach in simulations and real-world scenarios.

Multi-Modal MPPI and Active Inference for Reactive Task and Motion Planning

TL;DR

This work proposes a method for reactive TAMP to cope with runtime uncertainties and disturbances, which combines an Active Inference planner for adaptive high-level action selection and a novel Multi-Modal Model Predictive Path Integral controller (M3P2I) for low-level control.

Abstract

Task and Motion Planning (TAMP) has made strides in complex manipulation tasks, yet the execution robustness of the planned solutions remains overlooked. In this work, we propose a method for reactive TAMP to cope with runtime uncertainties and disturbances. We combine an Active Inference planner (AIP) for adaptive high-level action selection and a novel Multi-Modal Model Predictive Path Integral controller (M3P2I) for low-level control. This results in a scheme that simultaneously adapts both high-level actions and low-level motions. The AIP generates alternative symbolic plans, each linked to a cost function for M3P2I. The latter employs a physics simulator for diverse trajectory rollouts, deriving optimal control by weighing the different samples according to their cost. This idea enables blending different robot skills for fluid and reactive plan execution, accommodating plan adjustments at both the high and low levels to cope, for instance, with dynamic obstacles or disturbances that invalidate the current plan. We have tested our approach in simulations and real-world scenarios.
Paper Structure (24 sections, 23 equations, 7 figures, 2 tables, 4 algorithms)

This paper contains 24 sections, 23 equations, 7 figures, 2 tables, 4 algorithms.

Table of Contents

  1. Introduction
  2. Related work
  3. Contributions
  4. Background
  5. Active Inference Planner (AIP)
  6. Model Predictive Path Integral Control (MPPI)
  7. Methodology
  8. Overview
  9. Action planner - Active Inference Planner (AIP)
  10. Motion planner - Multi-Modal MPPI (M3P2I)
  11. Plan interface
  12. Experiments
  13. Push-pull scenario
  14. Action templates for AIP
  15. Cost functions for M3P2I
  16. ...and 9 more sections

Figures (7)

  • Figure 1: Proposed scheme. Given symbolic observations $o$ of the environment, the action planner computes $N$ different plan alternatives linked to individual cost functions $C_i$. M3P2I samples control input sequences and uses an importance sampling scheme to approximate the optimal control $u_0^*$.
  • Figure 2: Push-pull scenario. The dark purple object has to be placed on the green area. The robot can pull or push the object while avoiding dynamic and fixed obstacles. The objects and goals can have different initial positions.
  • Figure 3: Push and pull ideal configurations. The robot $R$ has to push or pull the object $O$ to the goal $G$.
  • Figure 4: Illustrative example of pulling and pushing a block to a goal. The strategy differs according to the object, goal location, and dynamic obstacle position. What action to perform is decided at runtime through multi-modal sampling.
  • Figure 5: Pick-place scenarios. The red cube has to be placed on top of the green cube. The red cube can be either on the table or a constrained shelf, requiring different pick strategies from the top or the side, respectively.
  • ...and 2 more figures