Decentralized Traffic Flow Optimization Through Intrinsic Motivation

TL;DR

The paper tackles decentralized traffic optimization under uncertainty by embedding empowerment-based intrinsic motivation into a fraction of autonomous vehicles within the Nagel-Schreckenberg model. It shows that agents maximizing local empowerment, without inter-vehicle communication, can yield substantial improvements in traffic flow, delay reduction, and a rightward shift of the critical density across a range of conditions. The approach avoids handcrafted reward functions and explicit coordination, highlighting the potential of intrinsic motivation for scalable, privacy-preserving traffic control. It also outlines directions for extending the framework to continuous limits and hybrid centralized–decentralized control architectures.

Abstract

Traffic congestion has long been an ubiquitous problem that is exacerbating with the rapid growth of megacities. In this proof-of-concept work we study intrinsic motivation, implemented via the empowerment principle, to control autonomous car behavior to improve traffic flow. In standard models of traffic dynamics, self-organized traffic jams emerge spontaneously from the individual behavior of cars, affecting traffic over long distances. Our novel car behavior strategy improves traffic flow while still being decentralized and using only locally available information without explicit coordination. Decentralization is essential for various reasons, not least to be able to absorb robustly substantial levels of uncertainty. Our scenario is based on the well-established traffic dynamics model, the Nagel-Schreckenberg cellular automaton. In a fraction of the cars in this model, we substitute the default behavior by empowerment, our intrinsic motivation-based method. This proposed model significantly improves overall traffic flow, mitigates congestion, and reduces the average traffic jam time.

Figures (6)

• Figure 1: Fundamental diagram of expected traffic flow for different values of the braking probability $p_{brake}$.
• Figure 2: 3-step empowerment, $\mathcal{E}^3(s)$, in bits for $p_{brake}=0.2$ and $\rho=0.2$
• Figure 3: Comparison of traffic flow with varying ratios of empowered agents, $0\%$ to $70\%$, across different vehicle densities on the x axis. It is evident that an increase in the horizon expands the range of densities for which the model remains effective. As expected, with an increase in $p_{brake}$ value, the flow decreases, but the effect of empowerment remains consistent. The analysis is done on a road of $L=1000$, $v_{max}=5$ with $T=5000$. Note, the curves intersect at nearby but not exactly the same point (cf., insert 'b')
• Figure 4: Comparison of the percentage (%) improvement in traffic flow for densities beyond critical density $(\rho_c)$ and those within the model's effective range, under varying ratios of empowered agents, different $p_{brake}$ values, and different empowerment planning horizons.
• Figure 5: Spatio-temporal view of traffic at densities corresponding to peak flow improvement achieved by our model. Each black dot represents a vehicle moving from left to right. The black streaks depict traffic jam waves propagating in reverse through the traffic. Increasing the number of empowered vehicles results in less dense jams and their quicker dissipation.