LITE: A Paradigm Shift in Multi-Object Tracking with Efficient ReID Feature Integration

TL;DR

The paper introduces LITE, a paradigm that integrates appearance feature extraction directly into the MOT pipeline using a standard detector (e.g., YOLOv8m), thereby removing separate ReID model inference and associated pre-/post-processing costs. Demonstrated with LITE:DeepSORT, this approach achieves real-time performance (e.g., 28 FPS) on MOT17 while maintaining competitive HOTA scores (~43) and significantly speeding up the tracking component compared to traditional DeepSORT, especially in crowded scenes. To ensure practical relevance, the authors deliver a holistic evaluation framework that measures the entire tracking pipeline—detection, ReID, and tracking—under real-time constraints and across diverse datasets, revealing common pitfalls in prior assessments. The work shows that LITE-applied trackers offer substantial speedups (2–4x) with minimal accuracy loss, suggesting broad applicability to real-time surveillance, autonomous systems, and action detection, with future work extending LITE to more trackers and edge-device deployment.

Abstract

The Lightweight Integrated Tracking-Feature Extraction (LITE) paradigm is introduced as a novel multi-object tracking (MOT) approach. It enhances ReID-based trackers by eliminating inference, pre-processing, post-processing, and ReID model training costs. LITE uses real-time appearance features without compromising speed. By integrating appearance feature extraction directly into the tracking pipeline using standard CNN-based detectors such as YOLOv8m, LITE demonstrates significant performance improvements. The simplest implementation of LITE on top of classic DeepSORT achieves a HOTA score of 43.03% at 28.3 FPS on the MOT17 benchmark, making it twice as fast as DeepSORT on MOT17 and four times faster on the more crowded MOT20 dataset, while maintaining similar accuracy. Additionally, a new evaluation framework for tracking-by-detection approaches reveals that conventional trackers like DeepSORT remain competitive with modern state-of-the-art trackers when evaluated under fair conditions. The code will be available post-publication at https://github.com/Jumabek/LITE.

Figures (6)

• Figure 1: Overview of SORT family Trackers, their methodologies, and comparison on the MOT17 train set. The figure shows the simplicity of LiteSORT (LITE+DeepSORT) as it replaces the external ReID CNN of DeepSORT with detection feature maps via the LITE paradigm.
• Figure 2: Efficient ReID feature extraction via the LITE paradigm.
• Figure 3: Sample frames from various datasets used in the analysis.
• Figure 5: Comparative analysis of detection settings' impact on HOTA scores across different input resolutions. Experiments conducted on the MOT17 dataset. LiteSORT is an alias for LITE:DeepSORT.
• Figure 6: Comparative analysis of detection settings' impact on FPS scores across different input resolutions. Experiments conducted on the MOT17 dataset. LiteSORT is an alias for LITE:DeepSORT.