DG-DETR: Toward Domain Generalized Detection Transformer

TL;DR

DG-DETR addresses domain generalization for DETR-based object detection by introducing two key components: WaveNP, a wavelet-guided style augmentation that perturbs only low-frequency domain-specific features to preserve object semantics, and DAQS, a domain-agnostic query selection mechanism that orthogonally projects queries away from style-prefixed axes to reduce domain biases. Implemented as a plug-and-play enhancement to DETR-based detectors, the approach yields improved out-of-distribution robustness across adverse weather and corruption benchmarks, including Diverse Weather Dataset (DWD) and Cityscapes-C, with substantial gains over baselines. The method is validated through ablations showing the necessity and complementarity of WaveNP and DAQS, and demonstrates compatibility with multiple DETR variants such as RT-DETR and DINO. Overall, DG-DETR offers a practical, effective strategy to extend DETR performance to unseen domains, enhancing reliability in real-world deployment.

Abstract

End-to-end Transformer-based detectors (DETRs) have demonstrated strong detection performance. However, domain generalization (DG) research has primarily focused on convolutional neural network (CNN)-based detectors, while paying little attention to enhancing the robustness of DETRs. In this letter, we introduce a Domain Generalized DEtection TRansformer (DG-DETR), a simple, effective, and plug-and-play method that improves out-of-distribution (OOD) robustness for DETRs. Specifically, we propose a novel domain-agnostic query selection strategy that removes domain-induced biases from object queries via orthogonal projection onto the instance-specific style space. Additionally, we leverage a wavelet decomposition to disentangle features into domain-invariant and domain-specific components, enabling synthesis of diverse latent styles while preserving the semantic features of objects. Experimental results validate the effectiveness of DG-DETR. Our code is available at https://github.com/sminhwang/DG-DETR.

Figures (5)

• Figure 1: Overview of the proposed DG-DETR framework. The backbone extracts multi-scale features, represented as $s_1$, $s_2$, and $s_3$, from its last three stages. These features are processed by the encoder and fed into the Domain-agnostic Query Selection (DAQS) module. The DAQS module explicitly removes domain-induced biases from object queries, providing high-quality initial queries for the decoder.
• Figure 2: Visualization of wavelet decomposition. The image is decomposed into low- and high-frequency components. In the Discrete Wavelet Transform (DWT) result, LL represents the low-frequency approximation, capturing the overall style. LH, HL, and HH represent the high-frequency detail coefficients (horizontal, vertical, and diagonal details, respectively), which primarily preserve object shapes and edge information.
• Figure 3: Overview of our style augmentation module. Wavelet transform decomposes the shallow CNN features into low- and high-frequency components. We perturb only the low-frequency features components.
• Figure 4: Visualization of the orthogonal projection. The query $Q$ is projected onto style-irrelevant semantic axes.
• Figure 5: Detection results on Night-Sunny, Dusk-Rainy, Night-Rainy, Daytime-Foggy. First Row: RT-DETR zhao2024detrs results. Second Row: The results with our method. Green bounding boxes indicate cars, and red bounding boxes indicate trucks.