Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Graph-based Adversarial Imitation Learning Framework for Reliable & Realtime Fleet Scheduling in Urban Air Mobility

Prithvi Poddar, Steve Paul, Souma Chowdhury

TL;DR

This work tackles reliable, real-time fleet scheduling for Urban Air Mobility under uncertainty by formulating a graph-based imitation learning framework. CapTAIN-GAIL uses Graph Capsule Convolutional Networks, a Transformer encoder, and a Multi-head Attention decoder to map rich state information into actions, trained via Generative Adversarial Imitation Learning with expert data generated by a Genetic Algorithm. In simulations with 8 vertiports and 40 eVTOLs, CapTAIN-GAIL achieves a higher mean profit and markedly better performance on unseen worst-case scenarios, while offering substantial speedups over GA. The study demonstrates that imitation learning can close the gap to optimal solutions and provide robust, scalable policies for real-time UAM fleet management.

Abstract

The advent of Urban Air Mobility (UAM) presents the scope for a transformative shift in the domain of urban transportation. However, its widespread adoption and economic viability depends in part on the ability to optimally schedule the fleet of aircraft across vertiports in a UAM network, under uncertainties attributed to airspace congestion, changing weather conditions, and varying demands. This paper presents a comprehensive optimization formulation of the fleet scheduling problem, while also identifying the need for alternate solution approaches, since directly solving the resulting integer nonlinear programming problem is computationally prohibitive for daily fleet scheduling. Previous work has shown the effectiveness of using (graph) reinforcement learning (RL) approaches to train real-time executable policy models for fleet scheduling. However, such policies can often be brittle on out-of-distribution scenarios or edge cases. Moreover, training performance also deteriorates as the complexity (e.g., number of constraints) of the problem increases. To address these issues, this paper presents an imitation learning approach where the RL-based policy exploits expert demonstrations yielded by solving the exact optimization using a Genetic Algorithm. The policy model comprises Graph Neural Network (GNN) based encoders that embed the space of vertiports and aircraft, Transformer networks to encode demand, passenger fare, and transport cost profiles, and a Multi-head attention (MHA) based decoder. Expert demonstrations are used through the Generative Adversarial Imitation Learning (GAIL) algorithm. Interfaced with a UAM simulation environment involving 8 vertiports and 40 aircrafts, in terms of the daily profits earned reward, the new imitative approach achieves better mean performance and remarkable improvement in the case of unseen worst-case scenarios, compared to pure RL results.

A Graph-based Adversarial Imitation Learning Framework for Reliable & Realtime Fleet Scheduling in Urban Air Mobility

TL;DR

This work tackles reliable, real-time fleet scheduling for Urban Air Mobility under uncertainty by formulating a graph-based imitation learning framework. CapTAIN-GAIL uses Graph Capsule Convolutional Networks, a Transformer encoder, and a Multi-head Attention decoder to map rich state information into actions, trained via Generative Adversarial Imitation Learning with expert data generated by a Genetic Algorithm. In simulations with 8 vertiports and 40 eVTOLs, CapTAIN-GAIL achieves a higher mean profit and markedly better performance on unseen worst-case scenarios, while offering substantial speedups over GA. The study demonstrates that imitation learning can close the gap to optimal solutions and provide robust, scalable policies for real-time UAM fleet management.

Abstract

The advent of Urban Air Mobility (UAM) presents the scope for a transformative shift in the domain of urban transportation. However, its widespread adoption and economic viability depends in part on the ability to optimally schedule the fleet of aircraft across vertiports in a UAM network, under uncertainties attributed to airspace congestion, changing weather conditions, and varying demands. This paper presents a comprehensive optimization formulation of the fleet scheduling problem, while also identifying the need for alternate solution approaches, since directly solving the resulting integer nonlinear programming problem is computationally prohibitive for daily fleet scheduling. Previous work has shown the effectiveness of using (graph) reinforcement learning (RL) approaches to train real-time executable policy models for fleet scheduling. However, such policies can often be brittle on out-of-distribution scenarios or edge cases. Moreover, training performance also deteriorates as the complexity (e.g., number of constraints) of the problem increases. To address these issues, this paper presents an imitation learning approach where the RL-based policy exploits expert demonstrations yielded by solving the exact optimization using a Genetic Algorithm. The policy model comprises Graph Neural Network (GNN) based encoders that embed the space of vertiports and aircraft, Transformer networks to encode demand, passenger fare, and transport cost profiles, and a Multi-head attention (MHA) based decoder. Expert demonstrations are used through the Generative Adversarial Imitation Learning (GAIL) algorithm. Interfaced with a UAM simulation environment involving 8 vertiports and 40 aircrafts, in terms of the daily profits earned reward, the new imitative approach achieves better mean performance and remarkable improvement in the case of unseen worst-case scenarios, compared to pure RL results.
Paper Structure (20 sections, 6 equations, 9 figures)

This paper contains 20 sections, 6 equations, 9 figures.

Table of Contents

  1. Introduction
  2. Related Works
  3. Problem Description and Formulation
  4. UAM Fleet Scheduling as an Integer Non-Linear Programming Problem
  5. Generating the Expert Demonstrations
  6. MDP Formulation
  7. Proposed Graph-based Adversarial Imitation Learning Approach
  8. Policy Network
  9. Training with Generative Adversarial Imitation Learning
  10. Experiments and Results
  11. Simulation Details
  12. Training Details
  13. Comparing CapTAIN vs. Genetic Algorithm
  14. Evaluating CapTAIN-GAIL
  15. Average Profits
  16. ...and 5 more sections

Figures (9)

  • Figure 1: A visual representation of the UAM fleet management problem. CapTAIN-GAIL is the imitation learning policy that takes as input, the information from the vertiport and eVTOL graphs along with operational costs, passenger demands, and air-corridor availabilities and decides the next vertiport that the decision-making agent should go to.
  • Figure 2: The learning framework for CapTAIN-GAIL
  • Figure 3: Difference in the profits earned by GA and CapTAIN, Positive values indicate better performance by GA and negative values indicate better performance by CapTAIN.
  • Figure 4: Comparing the average profits earned by all the 3 methods
  • Figure 5: Difference in the profits earned by CapTAIN-GAIL and CapTAIN, Positive values indicate better performance by CapTAIN-GAIL and negative values indicate better performance by CapTAIN.
  • ...and 4 more figures