HopTrack: A Real-time Multi-Object Tracking System for Embedded Devices

TL;DR

HopTrack is introduced, a real-time multi-object tracking system tailored for embedded devices that employs a novel discretized static and dynamic matching approach along with an innovative content-aware dynamic sampling technique to enhance tracking accuracy while meeting the real-time requirement.

Abstract

Multi-Object Tracking (MOT) poses significant challenges in computer vision. Despite its wide application in robotics, autonomous driving, and smart manufacturing, there is limited literature addressing the specific challenges of running MOT on embedded devices. State-of-the-art MOT trackers designed for high-end GPUs often experience low processing rates (<11fps) when deployed on embedded devices. Existing MOT frameworks for embedded devices proposed strategies such as fusing the detector model with the feature embedding model to reduce inference latency or combining different trackers to improve tracking accuracy, but tend to compromise one for the other. This paper introduces HopTrack, a real-time multi-object tracking system tailored for embedded devices. Our system employs a novel discretized static and dynamic matching approach along with an innovative content-aware dynamic sampling technique to enhance tracking accuracy while meeting the real-time requirement. Compared with the best high-end GPU modified baseline Byte (Embed) and the best existing baseline on embedded devices MobileNet-JDE, HopTrack achieves a processing speed of up to 39.29 fps on NVIDIA AGX Xavier with a multi-object tracking accuracy (MOTA) of up to 63.12% on the MOT16 benchmark, outperforming both counterparts by 2.15% and 4.82%, respectively. Additionally, the accuracy improvement is coupled with the reduction in energy consumption (20.8%), power (5%), and memory usage (8%), which are crucial resources on embedded devices. HopTrack is also detector agnostic allowing the flexibility of plug-and-play.

Figures (14)

• Figure 1: Performance comparison of HopTrack with baselines on embedded devices on MOT16. Circle size indicates the memory usage, while the number above the circle represents processing rate in fps.
• Figure 2: System overview of HopTrack. Hop Fuse associates active tracks with detections from dynamically sampled frames. Hop Update updates tracks' positions and suppresses inaccurate tracks.
• Figure 3: The yellow dashed box shows a prior detection, while the yellow box displays the current but incorrect tracking result. Blue boxes indicate candidate detections along the trajectory.
• Figure 4: Discretized static matching calculates the intensity distribution for each channel in every image cell and computes the Wasserstein distance for corresponding pairs of cells.
• Figure 5: Dynamic discretized matching uses feature vectors < R, G, B, $\mathbf{v_{x}}$, $\mathbf{v_{y}}$> for each image cell, followed by cosine similarity calculation to assess if two tracks represent the same object.