Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learning Hierarchical Control For Multi-Agent Capacity-Constrained Systems

Charlott Vallon, Alessandro Pinto, Bartolomeo Stellato, Francesco Borrelli

TL;DR

A data-driven hierarchical control scheme for a fleet of nonlinear, capacity-constrained autonomous agents in an iterative environment that leverages tools from iterative learning control to integrate learning at both hierarchy levels, and coordinate learning between levels to maintain closed-loop feasibility and performance improvement at each iteration.

Abstract

This paper introduces a novel data-driven hierarchical control scheme for managing a fleet of nonlinear, capacity-constrained autonomous agents in an iterative environment. We propose a control framework consisting of a high-level dynamic task assignment and routing layer and low-level motion planning and tracking layer. Each layer of the control hierarchy uses a data-driven Model Predictive Control (MPC) policy, maintaining bounded computational complexity at each calculation of a new task assignment or actuation input. We utilize collected data to iteratively refine estimates of agent capacity usage, and update MPC policy parameters accordingly. Our approach leverages tools from iterative learning control to integrate learning at both levels of the hierarchy, and coordinates learning between levels in order to maintain closed-loop feasibility and performance improvement of the connected architecture.

Learning Hierarchical Control For Multi-Agent Capacity-Constrained Systems

TL;DR

A data-driven hierarchical control scheme for a fleet of nonlinear, capacity-constrained autonomous agents in an iterative environment that leverages tools from iterative learning control to integrate learning at both hierarchy levels, and coordinate learning between levels to maintain closed-loop feasibility and performance improvement at each iteration.

Abstract

This paper introduces a novel data-driven hierarchical control scheme for managing a fleet of nonlinear, capacity-constrained autonomous agents in an iterative environment. We propose a control framework consisting of a high-level dynamic task assignment and routing layer and low-level motion planning and tracking layer. Each layer of the control hierarchy uses a data-driven Model Predictive Control (MPC) policy, maintaining bounded computational complexity at each calculation of a new task assignment or actuation input. We utilize collected data to iteratively refine estimates of agent capacity usage, and update MPC policy parameters accordingly. Our approach leverages tools from iterative learning control to integrate learning at both levels of the hierarchy, and coordinates learning between levels in order to maintain closed-loop feasibility and performance improvement of the connected architecture.
Paper Structure (24 sections, 3 theorems, 83 equations, 4 figures, 2 algorithms)

This paper contains 24 sections, 3 theorems, 83 equations, 4 figures, 2 algorithms.

Table of Contents

  1. Introduction
  2. Notation
  3. Problem Formulation
  4. Iterative Data-Driven Hierarchical Control
  5. High-level controller
  6. Low-level controller
  7. Hierarchical Control Architecture
  8. Properties
  9. Hierarchical LMPC
  10. Parameter Calculation
  11. Policy Formulation
  12. High-level policy $\pi^{H,r}$
  13. Low-level policy $\pi^{L,r}$
  14. Hierarchical LMPC Architecture
  15. Multi-Agent Formulation
  16. ...and 9 more sections

Key Result

Theorem 1

Consider system (eq:agentdyn) controlled by the low-level policy (eq:lowlevellmpc) during consecutive traversals of a single edge $e_{i,j}$ in $G$. Let $\mathcal{S}^{L,r}_{i,j}$ be the safe set corresponding to the edge $e_{i,j}$ at the start of iteration $r$ as defined in (eq:lowlevelss-update). Th

Figures (4)

  • Figure 1: Graph $G$ models tasks (nodes $v_i$) and connections between tasks (edges $e_{i,j}$). A fleet of $M$ agents initially at $v_1=D$ jointly completes as many tasks as possible while without depleting capacities before returning to $v_1=D$. A task assignment for $M=2$ agents is shown in red and blue.
  • Figure 2: With additional iterations, the high-level controller is able to improve on its initialization assignment (iteration $0$) and plan more productive task assignments, eventually converging to the task assignment shown in the bottom figure (iteration $5$).
  • Figure 3: The number of completed tasks increases during the first 3 iterations, before leveling out to a final task assignment.
  • Figure 4: At each traversal of the edge $e_{2,6}$, the low-level controller searches for a trajectory that minimizes the stage cost (\ref{['eq:examplestagecost']}) without increasing capacity state depletions along the edge. The controller is safely able to guide the agent to increased velocities at subsequent iterations, reducing the time required to traverse the edge while maintaining the agent's state of charge.

Theorems & Definitions (13)

  • Remark 1
  • Remark 2
  • Definition 1: Iterative Feasibility
  • Definition 2: Iterative Performance Improvement
  • Remark 3
  • Remark 4
  • Remark 5
  • Theorem 1
  • proof
  • Theorem 2: Iterative Feasibility
  • ...and 3 more