VissimRL: A Multi-Agent Reinforcement Learning Framework for Traffic Signal Control Based on Vissim

TL;DR

VissimRL introduces a modular RL framework for traffic signal control within the high-fidelity Vissim simulator, bridging academic RL research and industry practice. It combines a Python-based Vissim Wrapper with a standardized RL Environment Framework (Gymnasium/PettingZoo) to enable single- and multi-agent training while preserving realism. The approach demonstrates substantial development-effort reductions, maintains runtime efficiency, and shows consistent improvements in traffic metrics, including emergent green-wave coordination and real-world improvements at the Dayuan Interchange. This work positions RL-based traffic signal control as a viable option for high-fidelity simulations and lays groundwork for real-world deployment in intelligent transportation systems.

Abstract

Traffic congestion remains a major challenge for urban transportation, leading to significant economic and environmental impacts. Traffic Signal Control (TSC) is one of the key measures to mitigate congestion, and recent studies have increasingly applied Reinforcement Learning (RL) for its adaptive capabilities. With respect to SUMO and CityFlow, the simulator Vissim offers high-fidelity driver behavior modeling and wide industrial adoption but remains underutilized in RL research due to its complex interface and lack of standardized frameworks. To address this gap, this paper proposes VissimRL, a modular RL framework for TSC that encapsulates Vissim's COM interface through a high-level Python API, offering standardized environments for both single- and multi-agent training. Experiments show that VissimRL significantly reduces development effort while maintaining runtime efficiency, and supports consistent improvements in traffic performance during training, as well as emergent coordination in multi-agent control. Overall, VissimRL demonstrates the feasibility of applying RL in high-fidelity simulations and serves as a bridge between academic research and practical applications in intelligent traffic signal control.

Figures (9)

• Figure 1: Four typical signal phases at a two-lane intersection.
• Figure 5: Architecture of the VissimRL framework.
• Figure 6: Operation flow of the VissimRL environment framework with standard API interfaces.
• Figure 7: Synthetic traffic networks: (a) Single-Intersection and (b) Arterial-3, an arterial network with three intersections.
• Figure 8: Real-world scenario of the Dayuan Interchange: (a) aerial photograph and (b) corresponding simulation network.