Style Evolving along Chain-of-Thought for Unknown-Domain Object Detection

TL;DR

This work tackles Single-Domain Generalized Object Detection (Single-DGOD) by addressing the limitations of one-step textual prompts in capturing complex multi-style scenes. It introduces Style Evolving along Chain-of-Thought, a two-stage framework that first evolves style prompts from simple to refined descriptions using vocabularies and per-channel parameters $\mu_t$ and $\sigma_t$ learned through a consistency loss $\mathcal{L}_{tc}$, and then transfers style information via AdaIN on first-layer features. To preserve semantic content while evolving style, the method employs a Style Disentangled Module (with losses $\mathcal{L}_d$ and $\mathcal{L}_{sc}$) and a Class-Specific Prototype Clustering Module to bolster class-specific content representation. Across five adverse-weather scenarios and Reality-to-Art benchmarks, the approach achieves state-of-the-art generalization, outperforming one-step prompt baselines by substantial margins and demonstrating robust performance when paired with stronger backbones such as Swin. Collectively, this work advances robust zero-shot generalization in object detection by leveraging chain-of-thought driven style synthesis, explicit style-content disentanglement, and prototype-guided semantic supervision.

Abstract

Recently, a task of Single-Domain Generalized Object Detection (Single-DGOD) is proposed, aiming to generalize a detector to multiple unknown domains never seen before during training. Due to the unavailability of target-domain data, some methods leverage the multimodal capabilities of vision-language models, using textual prompts to estimate cross-domain information, enhancing the model's generalization capability. These methods typically use a single textual prompt, often referred to as the one-step prompt method. However, when dealing with complex styles such as the combination of rain and night, we observe that the performance of the one-step prompt method tends to be relatively weak. The reason may be that many scenes incorporate not just a single style but a combination of multiple styles. The one-step prompt method may not effectively synthesize combined information involving various styles. To address this limitation, we propose a new method, i.e., Style Evolving along Chain-of-Thought, which aims to progressively integrate and expand style information along the chain of thought, enabling the continual evolution of styles. Specifically, by progressively refining style descriptions and guiding the diverse evolution of styles, this approach enables more accurate simulation of various style characteristics and helps the model gradually learn and adapt to subtle differences between styles. Additionally, it exposes the model to a broader range of style features with different data distributions, thereby enhancing its generalization capability in unseen domains. The significant performance gains over five adverse-weather scenarios and the Real to Art benchmark demonstrate the superiorities of our method.

Figures (6)

• Figure 1: Style evolution along Chain-of-Thought involves a process that progresses from coarse to fine, continuously refining and expanding styles. (a) illustrates the style description guided by the Chain-of-Thought. (b) demonstrates how using text prompts ranging from simple to complex guides style evolution, enabling the model to progressively learn and adapt to subtle differences between styles.
• Figure 2: The overall architecture of the model. Leveraging contrastive loss and two consistency losses, we disentangle the first-layer features into content and style features. For style features, the CGSE module is employed to obtain style transfer parameters for style migration. Regarding content features, semantic enhancement is performed through class-specific prototypes. Ultimately, these two features are fused and input into the backbone network to generate features from layers 2 to 4. The output is then fed into the Region Proposal Network (RPN) for subsequent target localization and classification tasks.
• Figure 3: Complex Style Evolution: By using text prompts that progress from simplicity to complexity, the style is continuously evolved and expanded, thereby simulating a richer variety of style features with different data distributions. Parameters $\mu_t$ and $\sigma_t$ are trained using consistency loss, aligning the text features with the source domain visual features, and are subsequently utilized for style evolution.
• Figure 4: Qualitative Results: Detection results on different weather conditions. The first and second rows display the results from C-GAP C_Cap and our method. Objects highlighted in red boxes represent those missed by C-GAP C_Cap but correctly identified by our method.
• Figure 5: Visualization analysis of our method: The first column shows the model's detection results. The second column visualizes the features with style evolution using a one-step method, while the third and fourth columns represent style evolution guided by two-level and three-level chains of thought, respectively. The feature heatmaps show that as the chains of thought progress, the model increasingly focuses on foreground objects, validating the effectiveness of the method.