Real-time Lane-wise Traffic Monitoring in Optimal ROIs

TL;DR

This work tackles real-time highway traffic monitoring using PTZ cameras by automatically learning lane locations and directions from video, enabling lane-wise vehicle counting and traffic status estimation within optimal ROIs. It introduces an end-to-end system with Environment Learning to infer lanes from vehicle motion, a Camera View Checking module to detect view changes, and Road Condition/Traffic Status Detection for lane-level metrics. Key innovations include Video Rate-Computer Speed Synchronization, a CIOU-based, frame-skipping adaptation of DeepSort, and adaptive lane-ID assignment to achieve real-time, lane-focused analysis on a single GPU. The approach demonstrates competitive lane counting and robust traffic status detection across varied weather and lighting, highlighting practical utility for scalable highway ITS data collection and decision support.

Abstract

In the US, thousands of Pan, Tilt, and Zoom (PTZ) traffic cameras monitor highway conditions. There is a great interest in using these highway cameras to gather valuable road traffic data to support traffic analysis and decision-making for highway safety and efficient traffic management. However, there are too many cameras for a few human traffic operators to effectively monitor, so a fully automated solution is desired. This paper introduces a novel system that learns the locations of highway lanes and traffic directions from these camera feeds automatically. It collects real-time, lane-specific traffic data continuously, even adjusting for changes in camera angle or zoom. This facilitates efficient traffic analysis, decision-making, and improved highway safety.

Figures (6)

• Figure 1: (Top) Traditional System: Vehicles are detected, tracked, and counted across the entire frame, with manually defined counting lines, leading to suboptimal performance. (Bottom) Our system: Vehicle detection is performed across the entire frame. Vehicle tracking is concentrated in several automatically learned Regions of Interest (ROIs), optimizing detection and tracking results. Counting is lane-specific, with each vehicle assigned a unique LaneID, moving away from bi-directional counting. Best view in color.
• Figure 2: The system we designed is running on every local computer with a single GPU. This system gets real-time stream input from a single camera. Since cameras are not fixed on highways, the Environment Learning module will learn the road and lane information based on vehicle motion trajectories (the details can be found in our previous work: qiu2021intelligentqiu2022attentionqiu2024intelligent). When these parameters about road and lanes are learned for a particular view (can be defined by customer), the system will go to the next stage, to do Road Condition and Traffic Status Detection via each lane. An independent module ( i.e., Camera View Checking) keeps running all the time to check camera angle changes or not by comparing with the first, well-defined view. Once the system detects the camera angle/view is changed, the Road Condition and Traffic Status Detection module stops working immediately and the system goes to the Environment Learning to learn a new set of road and lane information parameters. More details are explained in Section \ref{['sec:method']}.
• Figure 3: Correct Camera View Change Detection Cases. (a) The cloud background of the right frame changed significantly from the left frame but did not cause false detection. The system did not report the camera view change. This is correct detection. (b) The left frame is taken at dawn and the right frame in the afternoon. The system did not report the camera view changes. This is correct detection.
• Figure 4: Wrong Camera View Change Detection Cases. (a) The left frame is taken in the morning and the right frame is taken at night. The algorithm gives false detection because of the flare in the second frame is background. The Hash value distance returned by our algorithm is larger than the threshold. This is wrong detection. (b) The algorithm predicts camera angle is changed in the right frame from the left frame while it is not. But it is very tough for humans as well to predict in this kind of scenario. This is wrong detection.
• Figure 5: 9 ITS traffic scenes recorded in sunny, rainy, snowy, nighttime, and congestion traffic conditions.