Balancing Stability and Plasticity in Pretrained Detector: A Dual-Path Framework for Incremental Object Detection

TL;DR

This work tackles the stability-plasticity dilemma in pretrained-model-based incremental object detection (PTMIOD), with a focus on cross-domain adaptation. It introduces a dual-path framework that freezes the localization pathway to preserve pretrained localization knowledge while enabling classification plasticity through LoRA-based fine-tuning and pseudo-feature replay, implemented in a DETR-based detector. The approach achieves state-of-the-art results on in-domain benchmarks (COCO, VOC) and strong cross-domain performance on TT100K, demonstrating robust cross-domain adaptation and sustained anti-forgetting capabilities. Overall, decoupling localization stability from classification plasticity within PTMIOD offers a practical and effective strategy for continual, privacy-friendly object detection.

Abstract

The balance between stability and plasticity remains a fundamental challenge in pretrained model-based incremental object detection (PTMIOD). While existing PTMIOD methods demonstrate strong performance on in-domain tasks aligned with pretraining data, their plasticity to cross-domain scenarios remains underexplored. Through systematic component-wise analysis of pretrained detectors, we reveal a fundamental discrepancy: the localization modules demonstrate inherent cross-domain stability-preserving precise bounding box estimation across distribution shifts-while the classification components require enhanced plasticity to mitigate discriminability degradation in cross-domain scenarios. Motivated by these findings, we propose a dual-path framework built upon pretrained DETR-based detectors which decouples localization stability and classification plasticity: the localization path maintains stability to preserve pretrained localization knowledge, while the classification path facilitates plasticity via parameter-efficient fine-tuning and resists forgetting with pseudo-feature replay. Extensive evaluations on both in-domain (MS COCO and PASCAL VOC) and cross-domain (TT100K) benchmarks show state-of-the-art performance, demonstrating our method's ability to effectively balance stability and plasticity in PTMIOD, achieving robust cross-domain adaptation and strong retention of anti-forgetting capabilities.

Figures (4)

• Figure 1: Overview of our dual-path PTMIOD framework, which decouples localization stability and classification plasticity, enabling robust adaptation in cross-domain scenarios.
• Figure 2: t-SNE visualization of category features on VOC (in-domain) and TT100K (cross-domain) datasets which are extracted by two models: original PTM (Original) and model after finetuned with PEFT (w/ PEFT).
• Figure 3: Overview of our dual-path PTMIOD framework built on DINO, with the DINO backbone omitted for clarity.
• Figure 4: Impact of different LoRA rank values on model plasticity, evaluated on the TT100K dataset.