FitLight: Federated Imitation Learning for Plug-and-Play Autonomous Traffic Signal Control

TL;DR

This work tackles the scalability and generalization problems of reinforcement learning for traffic signal control by introducing FitLight, a cloud-edge Federated Imitation Learning framework. It blends real-time imitation learning with reinforcement learning via a hybrid pressure representation, a MaxHP expert, and federated knowledge sharing across heterogeneous, pruned submodels to accelerate learning and enable deployment on ultra-low-resource devices. The approach yields faster convergence and strong starting performance, approaching or matching dynamic-duration baselines while offering robust plug-and-play capability across diverse multi-intersection networks. Practical validation on CityFlow with real and synthetic datasets demonstrates significant gains in efficiency, generalization, and deployability, including memory footprints around 15 KB per edge node and low communication overhead.

Abstract

Although Reinforcement Learning (RL)-based Traffic Signal Control (TSC) methods have been extensively studied, their practical applications still raise some serious issues such as high learning cost and poor generalizability. This is because the ``trial-and-error'' training style makes RL agents extremely dependent on the specific traffic environment, which also requires a long convergence time. To address these issues, we propose a novel Federated Imitation Learning (FIL)-based framework for multi-intersection TSC, named FitLight, which allows RL agents to plug-and-play for any traffic environment without additional pre-training cost. Unlike existing imitation learning approaches that rely on pre-training RL agents with demonstrations, FitLight allows real-time imitation learning and seamless transition to reinforcement learning. Due to our proposed knowledge-sharing mechanism and novel hybrid pressure-based agent design, RL agents can quickly find a best control policy with only a few episodes. Moreover, for resource-constrained TSC scenarios, FitLight supports model pruning and heterogeneous model aggregation, such that RL agents can work on a micro-controller with merely 16{\it KB} RAM and 32{\it KB} ROM. Extensive experiments demonstrate that, compared to state-of-the-art methods, FitLight not only provides a superior starting point but also converges to a better final solution on both real-world and synthetic datasets, even under extreme resource limitations.

Figures (4)

• Figure 1: Framework of FitLight.
• Figure 2: Architecture and workflow of FitLight.
• Figure 3: Comparison of convergence rates.
• Figure 4: Comparison of average travel time and reward.

Theorems & Definitions (3)

• Definition 1
• Definition 2
• Definition 3