Prompt to Transfer: Sim-to-Real Transfer for Traffic Signal Control with Prompt Learning

TL;DR

This work tackles the sim-to-real gap in traffic signal control by introducing PromptGAT, a prompt-driven grounded action transformation that embeds LLM-derived real-world dynamics into the GAT framework. By using in-context learning from LLMs to profile how weather, road type, and traffic state influence dynamics, and fusing this with a forward model, PromptGAT yields grounded actions that align simulated transitions with real-world behavior. The approach yields a more accurate forward model, enabling RL policies learned in simulation to transfer more effectively to real-world like environments, as demonstrated on CityFlow-SUMO setups with LibSignal. The findings show that PromptGAT reduces the transfer gap across multiple configurations and metrics, offering a practical route toward robust, deployable TSC systems that can adapt to diverse operational conditions.

Abstract

Numerous solutions are proposed for the Traffic Signal Control (TSC) tasks aiming to provide efficient transportation and mitigate congestion waste. In recent, promising results have been attained by Reinforcement Learning (RL) methods through trial and error in simulators, bringing confidence in solving cities' congestion headaches. However, there still exist performance gaps when simulator-trained policies are deployed to the real world. This issue is mainly introduced by the system dynamic difference between the training simulator and the real-world environments. The Large Language Models (LLMs) are trained on mass knowledge and proved to be equipped with astonishing inference abilities. In this work, we leverage LLMs to understand and profile the system dynamics by a prompt-based grounded action transformation. Accepting the cloze prompt template, and then filling in the answer based on accessible context, the pre-trained LLM's inference ability is exploited and applied to understand how weather conditions, traffic states, and road types influence traffic dynamics, being aware of this, the policies' action is taken and grounded based on realistic dynamics, thus help the agent learn a more realistic policy. We conduct experiments using DQN to show the effectiveness of the proposed PromptGAT's ability in mitigating the performance gap from simulation to reality (sim-to-real).

Figures (7)

• Figure 1: Integrating knowledge from LLMs into GAT. (a) LLMs have implicit human knowledge about the change in dynamics. (b) Comparisons between vanilla GAT and our proposed PromptGAT, with GAT integrating a prompt-based dynamics modeling module.
• Figure 2: An example of using LLM with a prompt template for answers to depict real-world dynamics by providing traffic state (vehicle number), weather type, and road type to induce the LLM to infer based on domain knowledge. Given the same vehicle quantity and road type, we could observe that the answers under different weathers abide by the reality situation that snowy weather is more severe than rainy weather.
• Figure 3: The overall framework of our proposed PromptGAT. (a) The structure of PromptGAT, with a prompt0based dynamics modeling integrating the knowledge of LLMs into the learning of forward model $f_{\phi^+}$. (b) Details of prompt-based dynamics modeling module that infer and integrate the change of dynamics with traffic states.
• Figure 4: Comparison of LLM prompt answers and real-world settings reflects the same tendency across 4 versions.
• Figure 5: The performance in the $E_{real}$ using Direct-Transfer and PromptGAT comparing to the performance in $E_{sim}$.