Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Train Till You Drop: Towards Stable and Robust Source-free Unsupervised 3D Domain Adaptation

Björn Michele, Alexandre Boulch, Tuan-Hung Vu, Gilles Puy, Renaud Marlet, Nicolas Courty

TL;DR

A sensible way to regularize the learning problem is proposed and a novel criterion based on agreement with a reference model is introduced, used to stop the training when appropriate and as validator to select hyperparameters without any knowledge on the target domain.

Abstract

We tackle the challenging problem of source-free unsupervised domain adaptation (SFUDA) for 3D semantic segmentation. It amounts to performing domain adaptation on an unlabeled target domain without any access to source data; the available information is a model trained to achieve good performance on the source domain. A common issue with existing SFUDA approaches is that performance degrades after some training time, which is a by product of an under-constrained and ill-posed problem. We discuss two strategies to alleviate this issue. First, we propose a sensible way to regularize the learning problem. Second, we introduce a novel criterion based on agreement with a reference model. It is used (1) to stop the training when appropriate and (2) as validator to select hyperparameters without any knowledge on the target domain. Our contributions are easy to implement and readily amenable for all SFUDA methods, ensuring stable improvements over all baselines. We validate our findings on various 3D lidar settings, achieving state-of-the-art performance. The project repository (with code) is: github.com/valeoai/TTYD.

Train Till You Drop: Towards Stable and Robust Source-free Unsupervised 3D Domain Adaptation

TL;DR

A sensible way to regularize the learning problem is proposed and a novel criterion based on agreement with a reference model is introduced, used to stop the training when appropriate and as validator to select hyperparameters without any knowledge on the target domain.

Abstract

We tackle the challenging problem of source-free unsupervised domain adaptation (SFUDA) for 3D semantic segmentation. It amounts to performing domain adaptation on an unlabeled target domain without any access to source data; the available information is a model trained to achieve good performance on the source domain. A common issue with existing SFUDA approaches is that performance degrades after some training time, which is a by product of an under-constrained and ill-posed problem. We discuss two strategies to alleviate this issue. First, we propose a sensible way to regularize the learning problem. Second, we introduce a novel criterion based on agreement with a reference model. It is used (1) to stop the training when appropriate and (2) as validator to select hyperparameters without any knowledge on the target domain. Our contributions are easy to implement and readily amenable for all SFUDA methods, ensuring stable improvements over all baselines. We validate our findings on various 3D lidar settings, achieving state-of-the-art performance. The project repository (with code) is: github.com/valeoai/TTYD.
Paper Structure (37 sections, 7 equations, 4 figures, 23 tables)

This paper contains 37 sections, 7 equations, 4 figures, 23 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. SFUDA in Computer Vision
  4. Mitigating the drift in SFUDA
  5. Method
  6. Training scheme
  7. Unsupervised stopping criterion and model validator
  8. Self-training module
  9. Experiments
  10. Experimental setup
  11. Stopping criterion TTYD$_\textit{stop}$
  12. Model validator TTYD$_\textit{valid}$
  13. 3D-SFUDA benchmark
  14. Application to image modality
  15. Ablations
  16. ...and 22 more sections

Figures (4)

  • Figure 1: Evolution of the performance of baselines without degradation prevention strategies as they train over 20k iterations. Our method (TTYD$_\textit{core}$) uses an unsupervised criterion to stop training. The horizontal dotted line illustrates that we keep the model obtained at the stopping point (marked with a cross). Models are trained on nuScenes (NS) and unsupervisedly adapted to SemanticKITTI (SK$_{10}$) and Waymo Open (WO$_{10}$).
  • Figure 2: Performance %mIoU (top), as reference, and class agreement in % (bottom), for training over 20k iterations. (1st column) the crosses indicate when TTYD$_\textit{stop}$ stops the training in different SFUDA setups. Dashed lines after the crosses just illustrate the expected degradation issue. In reality, we do not continue training once the criterion is triggered. (2nd and 3rd columns) the red curves correspond to the hyperparameters $\eta$ and $\lambda$ selected using TTYD$_\textit{valid}$ in NS$\rightarrow$SK$_{10}$, showing we pick the best ones.
  • Figure 3: Examples of results with TENT wangtent, SHOT shotliang20a and URMDA teja2021uncertainty on NS$\rightarrow$SK$_{10}$: ground truth (GT), initial model trained only on source data, best model as upper bound (using ground-truth knowledge of the target validation set), and "full" training for 20k iterations. "Ignore" points are removed for a better visualisation. Notable errors due to degradation are marked with a dashed rectangle.
  • Figure 4: Examples of results with TTYD$_\textit{stop}$: ground truth (GT), initial model trained only on source data, training with our training scheme when using our stopping criterion, and "full" training for 20k iterations. "Ignore" points are removed for a better visualisation. Notable errors due to degradation are marked with a dashed rectangle. Due to different class mappings, coloring can vary between the different settings.