Vehicle Perception from Satellite

TL;DR

A large-scale benchmark for traffic monitoring from satellite is built, which supports several tasks, including tiny object detection, counting and density estimation, and discusses the future prospects.

Abstract

Satellites are capable of capturing high-resolution videos. It makes vehicle perception from satellite become possible. Compared to street surveillance, drive recorder or other equipments, satellite videos provide a much broader city-scale view, so that the global dynamic scene of the traffic are captured and displayed. Traffic monitoring from satellite is a new task with great potential applications, including traffic jams prediction, path planning, vehicle dispatching, \emph{etc.}. Practically, limited by the resolution and view, the captured vehicles are very tiny (a few pixels) and move slowly. Worse still, these satellites are in Low Earth Orbit (LEO) to capture such high-resolution videos, so the background is also moving. Under this circumstance, traffic monitoring from the satellite view is an extremely challenging task. To attract more researchers into this field, we build a large-scale benchmark for traffic monitoring from satellite. It supports several tasks, including tiny object detection, counting and density estimation. The dataset is constructed based on 12 satellite videos and 14 synthetic videos recorded from GTA-V. They are separated into 408 video clips, which contain 7,336 real satellite images and 1,960 synthetic images. 128,801 vehicles are annotated totally, and the number of vehicles in each image varies from 0 to 101. Several classic and state-of-the-art approaches in traditional computer vision are evaluated on the datasets, so as to compare the performance of different approaches, analyze the challenges in this task, and discuss the future prospects. The dataset is available at: https://github.com/Chenxi1510/Vehicle-Perception-from-Satellite-Videos.

Figures (7)

• Figure 1: Examples of (a) satellite images captured over Dubai International Airport, and (b) the 1,214 vehicles in the scene highlighted by yellow points.
• Figure 2: Challenges in vehicle perception from satellite. The optical flow maps in (a) and (b) indicate the movements are complex and uneven. (c), (d), (e) and (f) display the noise in satellite videos. They are shelter, clouds, specularity and shadow from left to right.
• Figure 3: The annotation process of each real image in TMS. (a) is the original image. (b) shows the difference between consecutive frames. (c) is the difference of registered consecutive frames. (d) displays the final annotation of vehicles.
• Figure 4: Image distribution of different datasets over the number of vehicles. Note that TMS-Real and TMS-Game represent the real and synthetic parts of TMS, respectively.
• Figure 5: Vehicle density distribution of TMS-real (a) and TMS-game (b).