Locating and Mitigating Gradient Conflicts in Point Cloud Domain Adaptation via Saliency Map Skewness

TL;DR

Point-cloud unsupervised domain adaptation often suffers from gradient conflicts between the supervised source task and self-supervised auxiliary tasks. The authors introduce SM-DSB, a saliency-map skewness-based data sampling block that implicitly estimates gradient conflicts without target labels and gates SSL contributions per sample within a two-step MTL framework. The method is lightweight and broadly compatible with mainstream point-cloud DA models, delivering consistent gains and state-of-the-art results on PointDA-10 when combined with Self-dist-GCN, along with insights from backpropagation analysis. This work offers a practical mechanism to reduce negative transfer and advances understanding of gradient dynamics in cross-domain learning.

Abstract

Object classification models utilizing point cloud data are fundamental for 3D media understanding, yet they often struggle with unseen or out-of-distribution (OOD) scenarios. Existing point cloud unsupervised domain adaptation (UDA) methods typically employ a multi-task learning (MTL) framework that combines primary classification tasks with auxiliary self-supervision tasks to bridge the gap between cross-domain feature distributions. However, our further experiments demonstrate that not all gradients from self-supervision tasks are beneficial and some may negatively impact the classification performance. In this paper, we propose a novel solution, termed Saliency Map-based Data Sampling Block (SM-DSB), to mitigate these gradient conflicts. Specifically, our method designs a new scoring mechanism based on the skewness of 3D saliency maps to estimate gradient conflicts without requiring target labels. Leveraging this, we develop a sample selection strategy that dynamically filters out samples whose self-supervision gradients are not beneficial for the classification. Our approach is scalable, introducing modest computational overhead, and can be integrated into all the point cloud UDA MTL frameworks. Extensive evaluations demonstrate that our method outperforms state-of-the-art approaches. In addition, we provide a new perspective on understanding the UDA problem through back-propagation analysis.

Figures (11)

• Figure 1: The left figure illustrates the design principle of our methods: we first use saliency map skewness to estimate the gradient conflict and then identify negative self-supervision gradient, optimize the sum gradient, making it closer to Oracle gradients. The right figure shows the correlation analysis between the saliency map skewness and gradient conflicts. Each point represents an observation from a specific training step, with its position along the X and Y axes indicating its values for the saliency map skewness and cosine similarity between $G_{SSL}$ and $G_{oracle}$. Lower skewness indicates less conflict and more gain in the cousin similarity.
• Figure 2: Overview of our SM-DSB plugged into the MTL framework. The measurer computes the skewness of instance-level saliency maps to estimate the gradient conflict for each sample and passes the result to the selector. Then, the selector filters out samples that are suitable for self-supervised training instead of accepting them all.
• Figure 3: Confusion matrices of classifying testing samples on S* → M, the left and right correspond the methods without and with our SM-DSB under the Self-dist GCN framework.
• Figure 4: Conflict inconsistency evaluated on S$\to$M. The figure on the left shows the cosine similarity between aggregated gradients across tasks and the oracle gradients. The figure on the right represents the cosine similarity of the inter-task gradient between the self-supervision task and the gradient of the classification task.
• Figure 5: On the left is a table with a high skewness score, while on the right is a chair with a low skewness score. The red region indicates the salient area, whereas the pink and blue regions are less significant. High skewness (right-skewed) signifies a concentration of a few critical points, making the model more sensitive to input disturbances and potentially less robust. Conversely, low skewness (left-skewed) shows more dispersed attention, which indicates the model's ability to maintain classification performance under domain changes.