Decision Boundary Optimization-Informed Domain Adaptation

TL;DR

This work addresses the limitation of traditional MMD-based domain adaptation methods that prioritize distribution alignment over decision-boundary optimization. It introduces Decision Boundary optimization-informed MMD (DB-MMD), which jointly concentrates on marginal/conditional distribution alignment, cross-sub-domain discriminativeness, and a decision-boundary aware graph to tighten intra-class and separate inter-class cross-domain samples. By embedding DB-MMD into baseline DA models (e.g., CDDA, DGA-DA, MEDA), the approach yields consistent improvements across eight standard DA datasets, with gains up to 9.5 percentage points in some tasks and competitive performance on others. The framework supports kernelization and provides insights into parameter sensitivity and convergence, highlighting its practical impact for boundary-aware domain adaptation in real-world applications.

Abstract

Maximum Mean Discrepancy (MMD) is widely used in a number of domain adaptation (DA) methods and shows its effectiveness in aligning data distributions across domains. However, in previous DA research, MMD-based DA methods focus mostly on distribution alignment, and ignore to optimize the decision boundary for classification-aware DA, thereby falling short in reducing the DA upper error bound. In this paper, we propose a strengthened MMD measurement, namely, Decision Boundary optimization-informed MMD (DB-MMD), which enables MMD to carefully take into account the decision boundaries, thereby simultaneously optimizing the distribution alignment and cross-domain classifier within a hybrid framework, and leading to a theoretical bound guided DA. We further seamlessly embed the proposed DB-MMD measurement into several popular DA methods, e.g., MEDA, DGA-DA, to demonstrate its effectiveness w.r.t different experimental settings. We carry out comprehensive experiments using 8 standard DA datasets. The experimental results show that the DB-MMD enforced DA methods improve their baseline models using plain vanilla MMD, with a margin that can be as high as 9.5.

Figures (12)

• Figure 1: Fig.\ref{['fig:1']}.(a) shows that the source domain and the target domain samples depict a large domain divergence in the original feature space. Fig.\ref{['fig:1']}.(b) highlights that distribution alignment-based DA drags close the domains and the sub-domains but tends to ignore to optimize the decision boundary for yielding the classifier optimization ensured functional learning as required in Fig.\ref{['fig:1']}.(c).
• Figure 2: Illustration of the proposed decision boundary optimization-informed DA (DB-DA). Fig.\ref{['fig:2']} (a): the original source and target domain distributions; Fig.\ref{['fig:2']} (b,c) illustrate DB-DA aligning cross-domain distributions closely yet discriminatively by using MMD. Fig.\ref{['fig:2']} (d) shows the proposed DA aware of decision boundary through the specifically designed 'compacting graph' and 'separation graph'; Fig.\ref{['fig:2']} (e) illustrates the achieved latent joint subspace where both marginal and class conditional data distributions are aligned discriminatively and the decision boundaries are clearly optimized.
• Figure 3: In Fig.\ref{['fig:graph1']}.(a), DA explores the effectiveness of distribution alignment to drag close the domains and the sub-domains, while ignoring to optimize the samples lying around decision boundaries (Fig.\ref{['fig:graph1']}.(b,c)) for generating a decision boundary optimization guaranteed functional learning as illustrated in Fig.\ref{['fig:graph1']}.(d).
• Figure 4: Sample images from 8 datasets used in our experiments. Each dataset represents a different domain. The OFFICE dataset contains three sub-datasets: DSLR, Amazon, and Webcam.
• Figure 5: In Fig.\ref{['fig:DB']}.(a), the red portion denotes the baseline model of JDA, which is further improved by CDDA in hybridizing the repulse force term formalized in the yellow part. Then, based on CDDA, a geometric regularization is also embedded to formalize the DGA-DA. Fig.\ref{['fig:DB']}.(b) illustrates the derived DA models based on the three baseline models in Fig.\ref{['fig:DB']}.(a).