DRIVE: Dual-Robustness via Information Variability and Entropic Consistency in Source-Free Unsupervised Domain Adaptation

TL;DR

The paper tackles Source-Free Unsupervised Domain Adaptation (SFUDA), where a pretrained model must adapt to a labeled-free target domain using only unlabeled target data and without access to source data. It introduces DRIVE, a dual-model framework that leverages mutual-information-based consistency and entropy-aware pseudo-labeling, organized into two stages: Stage 1 aligns stable features via a dual-model ViL setup, and Stage 2 dynamically guides perturbations to explore target variability while preserving known mappings. Key contributions include a dual-model architecture that balances consistency and exploration, an entropy-aware labeling scheme to focus on reliable samples, and a dynamic perturbation strategy that calibrates exploration based on Stage 1 findings, all supported by ablations and Grad-CAM analyses. Empirically, DRIVE delivers improved adaptation accuracy and robustness acrossOffice-31, Office-Home, and DomainNet-126 benchmarks, indicating strong potential for real-world domain transfer in cluttered, noisy, or privacy-constrained settings.

Abstract

Adapting machine learning models to new domains without labeled data, especially when source data is inaccessible, is a critical challenge in applications like medical imaging, autonomous driving, and remote sensing. This task, known as Source-Free Unsupervised Domain Adaptation (SFUDA), involves adapting a pre-trained model to a target domain using only unlabeled target data, which can lead to issues such as overfitting, underfitting, and poor generalization due to domain discrepancies and noise. Existing SFUDA methods often rely on single-model architectures, struggling with uncertainty and variability in the target domain. To address these challenges, we propose DRIVE (Dual-Robustness through Information Variability and Entropy), a novel SFUDA framework leveraging a dual-model architecture. The two models, initialized with identical weights, work in parallel to capture diverse target domain characteristics. One model is exposed to perturbations via projection gradient descent (PGD) guided by mutual information, focusing on high-uncertainty regions. We also introduce an entropy-aware pseudo-labeling strategy that adjusts label weights based on prediction uncertainty, ensuring the model focuses on reliable data while avoiding noisy regions. The adaptation process has two stages: the first aligns the models on stable features using a mutual information consistency loss, and the second dynamically adjusts the perturbation level based on the loss from the first stage, encouraging the model to explore a broader range of the target domain while preserving existing performance. This enhances generalization capabilities and robustness against interference. Evaluations on standard SFUDA benchmarks show that DRIVE consistently outperforms previous methods, delivering improved adaptation accuracy and stability across complex target domains.

Figures (2)

• Figure 1: Overview of our DRIVE: it illustrates the two-stage adaptation process of DRIVE, including task-specific ViL model customization and knowledge adaptation with perturbed model encouragement, enhancing robustness and generalization in SFUDA.
• Figure 2: Grad-CAM visualization of our proposed method (DRIVE) compared to the primary baseline (DIFO), and the evolution of Grad-CAM visualizations for DRIVE as the number of iterations increases.