EW-DETR: Evolving World Object Detection via Incremental Low-Rank DEtection TRansformer

TL;DR

This work proposes EW-DETR framework that augments DETR-based detectors with three synergistic modules: Incremental LoRA Adapters for exemplar-free incremental learning under evolving domains; a Query-Norm Objectness Adapter that decouples objectness-aware features from DETR decoder queries; and Entropy-Aware Unknown Mixing for calibrated unknown detection.

Abstract

Real-world object detection must operate in evolving environments where new classes emerge, domains shift, and unseen objects must be identified as "unknown": all without accessing prior data. We introduce Evolving World Object Detection (EWOD), a paradigm coupling incremental learning, domain adaptation, and unknown detection under exemplar-free constraints. To tackle EWOD, we propose EW-DETR framework that augments DETR-based detectors with three synergistic modules: Incremental LoRA Adapters for exemplar-free incremental learning under evolving domains; a Query-Norm Objectness Adapter that decouples objectness-aware features from DETR decoder queries; and Entropy-Aware Unknown Mixing for calibrated unknown detection. This framework generalises across DETR-based detectors, enabling state-of-the-art RF-DETR to operate effectively in evolving-world settings. We also introduce FOGS (Forgetting, Openness, Generalisation Score) to holistically evaluate performance across these dimensions. Extensive experiments on Pascal Series and Diverse Weather benchmarks show EW-DETR outperforms other methods, improving FOGS by 57.24%.

Figures (8)

• Figure 1: Overview of Evolving World Object Detection (EWOD).Top (Training): Sequential tasks introduce disjoint classes $\mathcal{K}_t$ across shifting domains $\mathcal{D}_t$ (Day $\to$ Night $\to$ Fog). Bottom (Deployment): The detector must (i) identify unseen objects $\mathcal{U}_t$ without supervision; (ii) retain knowledge of all prior classes across all domains; (iii) incrementally learn former unknowns when their labels are revealed later (e.g., $\mathcal{U}_2 \to \mathcal{K}_3$) and (iv) achieve all these objectives without revisiting any previous data. Best viewed in colour with zoom.
• Figure 2: Top Row: gives an overview of the EW-DETR framework, which augments the standard DETR pipeline with three key modules: Incremental LoRA Adapters for exemplar-free incremental learning under evolving domains, Query-Norm Objectness Adapter for decoupled objectness-aware features, and Entropy-Aware Unknown Mixing for calibrated unknown detection. Bottom Row: illustrates a quick qualitative comparison of EWOD with other OD approaches. Best viewed in colour with zoom.
• Figure 3: EW-DETR framework for Evolving-World Object Detection. An input image is processed by a frozen backbone and transformer encoder–decoder equipped with aggregate $(\Delta \mathbf{W}^{t-1}_{\text{agg}})$ and task-specific $(\Delta \mathbf{W}^{t}_{\text{task}})$ LoRA adapters, yielding class-agnostic query features. These queries are reparameterised by the Query-Norm Objectness Adapter and passed through a classification head and an objectness head, whose outputs are combined by Entropy-aware Unknown Mixing module to produce calibrated class scores, while a localisation head predicts bounding boxes to form final detections across tasks.
• Figure 4: t-SNE visualisation of decoder query features comparing EW-DETR with recent methods on VOC [1:10] $\to$ Clipart [11:18] task. The top row shows features obtained during inference on VOC [1:10] classes, while the bottom row shows features evaluated on the combined test set comprising VOC [1:18] and Clipart [1:18] classes. Notably, EW-DETR is only able to preserve distinct class clusters even under severe domain shift (VOC $\to$ Clipart) while also generalising to unseen class-domain pairs: VOC [11:18] and Clipart [1:10], whereas other recent methods exhibit severe feature collapse in $\mathcal{T}_2$. Best viewed in colour with zoom.
• Figure S1: Effect of \ref{['fig:data_aware_beta']} data-aware vs. \ref{['fig:fixed_beta']} fixed merging.