An In-Depth Analysis of Adversarial Discriminative Domain Adaptation for Digit Classification

TL;DR

This work investigates domain shift in digit classification by implementing Adversarial Discriminative Domain Adaptation (ADDA) in a fully reproducible setting and evaluating across all six domain shifts formed by MNIST, USPS, and SVHN. ADDA trains a source encoder and classifier, then learns a target encoder via a domain-adversarial loss to map into a shared feature space with unshared weights, followed by testing with the source classifier. The results show that ADDA generally improves out-of-domain accuracy in five of six domain shifts, with MNIST-related shifts driving the largest gains, while in-domain accuracy often declines, especially for SVHN, suggesting limits when domain shifts are highly complex. Qualitative analyses using confusion matrices and t-SNE plots provide insight into why ADDA succeeds or fails in specific shifts and motivate future enhancements such as explainability methods and domain randomization to bolster generalization.

Abstract

Domain adaptation is an active area of research driven by the growing demand for robust machine learning models that perform well on real-world data. Adversarial learning for deep neural networks (DNNs) has emerged as a promising approach to improving generalization ability, particularly for image classification. In this paper, we implement a specific adversarial learning technique known as Adversarial Discriminative Domain Adaptation (ADDA) and replicate digit classification experiments from the original ADDA paper. We extend their findings by examining a broader range of domain shifts and provide a detailed analysis of in-domain classification accuracy post-ADDA. Our results demonstrate that ADDA significantly improves accuracy across certain domain shifts with minimal impact on in-domain performance. Furthermore, we provide qualitative analysis and propose potential explanations for ADDA's limitations in less successful domain shifts. Code is at https://github.com/eugenechoi2004/COS429_FINAL .

Figures (9)

• Figure 1: Images from the MNIST, SVHN, and USPS datasets of the digit $3$
• Figure 2: MNIST $\rightarrow$ SVHN domain shift confusion matrices
• Figure 3: SVHN $\rightarrow$ MNIST domain shift confusion matrices
• Figure 4: SVHN $\rightarrow$ USPS domain shift confusion matrices
• Figure 5: USPS $\rightarrow$ SVHN domain shift confusion matrices