Intelligent Traffic Surveillance for Real-Time Vehicle Detection, License Plate Recognition, and Speed Estimation

TL;DR

This work tackles speeding-related road fatalities in developing regions by presenting a real-time, end-to-end traffic surveillance pipeline for Uganda that jointly performs vehicle detection, license plate recognition, and speed estimation, with automated ticketing via SMS. It collects a multi-device dataset (speed gun, Canon camera, mobile phone), uses YOLOv8 for vehicle detection, ByteTrack for tracking, CNN and Transformer OCR (TrOCR) for plate transcription, and perspective-transform-based speed estimation, culminating in automated enforcement through Africa's Talking. Key results include a license-plate detection mAP of $0.979$, OCR CERs of $0.0385$ (CNN) and $0.0179$ (Transformer), and speed estimates within $±10$ km/h, demonstrating practical viability in low-resource settings. The proposed unified framework offers a path to scalable, real-time traffic enforcement in Uganda and similar contexts, with potential to reduce road accidents when deployed at scale.

Abstract

Speeding is a major contributor to road fatalities, particularly in developing countries such as Uganda, where road safety infrastructure is limited. This study proposes a real-time intelligent traffic surveillance system tailored to such regions, using computer vision techniques to address vehicle detection, license plate recognition, and speed estimation. The study collected a rich dataset using a speed gun, a Canon Camera, and a mobile phone to train the models. License plate detection using YOLOv8 achieved a mean average precision (mAP) of 97.9%. For character recognition of the detected license plate, the CNN model got a character error rate (CER) of 3.85%, while the transformer model significantly reduced the CER to 1.79%. Speed estimation used source and target regions of interest, yielding a good performance of 10 km/h margin of error. Additionally, a database was established to correlate user information with vehicle detection data, enabling automated ticket issuance via SMS via Africa's Talking API. This system addresses critical traffic management needs in resource-constrained environments and shows potential to reduce road accidents through automated traffic enforcement in developing countries where such interventions are urgently needed.

Figures (12)

• Figure 1: Overview of the flow system pipeline.
• Figure 2: TruCam II Clip Viewer interface displaying captured vehicle snapshot details. The measured speed is 39 Km/h.
• Figure 3: A frame from a video is used for vehicle detection; the output is then passed to the plate detection, whereby the characters are extracted using a recognition model.
• Figure 4: The source and target regions of the actual road scene. Points ABCD show the stretch of road as captured, while points A*B*C*D* show the transformed scene. This was crucial for speed estimation.
• Figure 5: Training curves for the license plate detection model over 50 epochs, showcasing the progress of box loss, classification loss, detection filter loss, and mean Average Precision.