Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Latent Distillation for Continual Object Detection at the Edge

Francesco Pasti, Marina Ceccon, Davide Dalle Pezze, Francesco Paissan, Elisabetta Farella, Gian Antonio Susto, Nicola Bellotto

TL;DR

The paper tackles distribution shifts in object detection on resource-constrained edge devices by evaluating a lightweight detector (NanoDet) for continual learning and introducing Latent Distillation (LD) to reduce update cost. LD shares lower, frozen layers between teacher and student, propagating latent representations to upper layers and applying distillation only to old classes, thereby cutting memory and FLOPs. Across VOC and COCO benchmarks, LD achieves competitive accuracy with substantial overhead reductions (74% fewer distillation parameters and 56% fewer FLOPs) compared to traditional distillation methods, while SID often provides the strongest stability for multi-class tasks. The work demonstrates the practicality of edge-friendly continual learning for one-stage detectors and offers a path toward real-time edge adaptation in dynamic environments.

Abstract

While numerous methods achieving remarkable performance exist in the Object Detection literature, addressing data distribution shifts remains challenging. Continual Learning (CL) offers solutions to this issue, enabling models to adapt to new data while maintaining performance on previous data. This is particularly pertinent for edge devices, common in dynamic environments like automotive and robotics. In this work, we address the memory and computation constraints of edge devices in the Continual Learning for Object Detection (CLOD) scenario. Specifically, (i) we investigate the suitability of an open-source, lightweight, and fast detector, namely NanoDet, for CLOD on edge devices, improving upon larger architectures used in the literature. Moreover, (ii) we propose a novel CL method, called Latent Distillation~(LD), that reduces the number of operations and the memory required by state-of-the-art CL approaches without significantly compromising detection performance. Our approach is validated using the well-known VOC and COCO benchmarks, reducing the distillation parameter overhead by 74\% and the Floating Points Operations~(FLOPs) by 56\% per model update compared to other distillation methods.

Latent Distillation for Continual Object Detection at the Edge

TL;DR

The paper tackles distribution shifts in object detection on resource-constrained edge devices by evaluating a lightweight detector (NanoDet) for continual learning and introducing Latent Distillation (LD) to reduce update cost. LD shares lower, frozen layers between teacher and student, propagating latent representations to upper layers and applying distillation only to old classes, thereby cutting memory and FLOPs. Across VOC and COCO benchmarks, LD achieves competitive accuracy with substantial overhead reductions (74% fewer distillation parameters and 56% fewer FLOPs) compared to traditional distillation methods, while SID often provides the strongest stability for multi-class tasks. The work demonstrates the practicality of edge-friendly continual learning for one-stage detectors and offers a path toward real-time edge adaptation in dynamic environments.

Abstract

While numerous methods achieving remarkable performance exist in the Object Detection literature, addressing data distribution shifts remains challenging. Continual Learning (CL) offers solutions to this issue, enabling models to adapt to new data while maintaining performance on previous data. This is particularly pertinent for edge devices, common in dynamic environments like automotive and robotics. In this work, we address the memory and computation constraints of edge devices in the Continual Learning for Object Detection (CLOD) scenario. Specifically, (i) we investigate the suitability of an open-source, lightweight, and fast detector, namely NanoDet, for CLOD on edge devices, improving upon larger architectures used in the literature. Moreover, (ii) we propose a novel CL method, called Latent Distillation~(LD), that reduces the number of operations and the memory required by state-of-the-art CL approaches without significantly compromising detection performance. Our approach is validated using the well-known VOC and COCO benchmarks, reducing the distillation parameter overhead by 74\% and the Floating Points Operations~(FLOPs) by 56\% per model update compared to other distillation methods.
Paper Structure (14 sections, 3 figures, 4 tables, 1 algorithm)

This paper contains 14 sections, 3 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Works
  3. CLOD at the Edge
  4. Efficient Object Detection Pipeline
  5. Latent Distillation
  6. Experimental Setting
  7. Dataset and Continual Learning Scenario
  8. Evaluation Metrics and Implementation Details
  9. Definition of Compared Methods
  10. Results and Discussion
  11. Multiple Classes
  12. One Class
  13. Sequential One Class
  14. Conclusion and Future Works

Figures (3)

  • Figure 1: Left: Continual Learning setting for Object Detection. From one task to the next, the model must learn from a new dataset where only new classes are annotated. The model is expected to learn new classes without compromising performance on previous ones as depicted in the bottom row. Right: Open-source NanoDet architecture. The image is fed through the backbone, three of its hidden representations are used by the GhostPAN feature pyramid and later fed to the lightweight detection heads.
  • Figure 2: Left: Classic distillation, the teacher is frozen (in blue) and distills knowledge to the student. In this setting, the memory occupation of the CL strategy is double the amount of parameters of the detection network. Right: Latent Distillation, student and teacher share a common frozen part. This enables reducing the memory and computation of the CL strategy.
  • Figure 3: Left: Results on the Sequential One Class (15p1) scenario. Right: Overall mAP vs total parameters required by the CL method on the 19p1 scenario. Our method shows the best overall performances when considering the two metrics.