TopView: Vectorising road users in a bird's eye view from uncalibrated street-level imagery with deep learning

TL;DR

This work introduces TopView, a calibration-free framework that learns a scene vanishing point to transform 2D road-user detections into a vector BEV map with 3D bounding boxes and temporal trajectories. It integrates an VP estimator, a YOLOv5m-based detector with DeepSort tracking, a geometric 2D-to-3D transformation, and a homography-based BEV mapping, enabling geolocated urban analytics from uncalibrated street imagery. The VP regression uses a logcosh loss, while object localization optimizes a combined $L_{loc}$ and $L_{conf}$ objective, yielding robust performance across daylight, night, and adverse weather. TopView demonstrates practical applications including city-scale occupancy visualization, social-distancing exposure analysis, and an anonymized, token-based representation of moving road users, with potential for ABM integration and privacy-preserving surveillance. Overall, the method broadens BEV applicability to uncalibrated data, supports vectorized outputs suitable for urban modeling, and offers scalable, privacy-conscious insights for traffic management and planning.

Abstract

Generating a bird's eye view of road users is beneficial for a variety of applications, including navigation, detecting agent conflicts, and measuring space occupancy, as well as the ability to utilise the metric system to measure distances between different objects. In this research, we introduce a simple approach for estimating a bird's eye view from images without prior knowledge of a given camera's intrinsic and extrinsic parameters. The model is based on the orthogonal projection of objects from various fields of view to a bird's eye view by learning the vanishing point of a given scene. Additionally, we utilised the learned vanishing point alongside the trajectory line to transform the 2D bounding boxes of road users into 3D bounding information. The introduced framework has been applied to several applications to generate a live Map from camera feeds and to analyse social distancing violations at the city scale. The introduced framework shows a high validation in geolocating road users in various uncalibrated cameras. It also paves the way for new adaptations in urban modelling techniques and simulating the built environment accurately, which could benefit Agent-Based Modelling by relying on deep learning and computer vision.

Figures (10)

• Figure 1: Examples of different sites of various street layouts and their estimated BEV map on a Google Map.
• Figure 2: The overall architecture of temporal localisation and spatial representation of scene objects from video streams.
• Figure 3: Estimating 3D bounding boxes from 2D bounding boxes from different poses of a given road user.
• Figure 4: Transforming a given image input to a Bird’s eye view.
• Figure 5: Sample of estimated VP points in the various dataset types (Predicted point in red dot, ground truth in blue dot). a given image input to a Bird’s eye view.