CMD: Constraining Multimodal Distribution for Domain Adaptation in Stereo Matching

TL;DR

This work addresses the domain shift in stereo matching, where unseen target domains often produce multimodal disparity distributions that degrade generalization. It introduces CMD, a framework combining uncertainty-regularized minimization ($L_u$) and anisotropic soft argmin to constrain the target-domain disparity distribution to be unimodal and sharp, boosting cross-domain performance. The approach is compatible with multiple stereo networks and further enhanced by pseudo-label self-distillation, achieving consistent improvements across synthetic-to-real adaptation on KITTI and ETH3D datasets. By quantifying distribution sharpness with metrics such as MSM, PER, and Entropy, and by sharpening the disparity distribution with a temperature-controlled softmax, CMD delivers significant gains without requiring labeled target-domain data, making it practical for real-world deployment.

Abstract

Recently, learning-based stereo matching methods have achieved great improvement in public benchmarks, where soft argmin and smooth L1 loss play a core contribution to their success. However, in unsupervised domain adaptation scenarios, we observe that these two operations often yield multimodal disparity probability distributions in target domains, resulting in degraded generalization. In this paper, we propose a novel approach, Constrain Multi-modal Distribution (CMD), to address this issue. Specifically, we introduce \textit{uncertainty-regularized minimization} and \textit{anisotropic soft argmin} to encourage the network to produce predominantly unimodal disparity distributions in the target domain, thereby improving prediction accuracy. Experimentally, we apply the proposed method to multiple representative stereo-matching networks and conduct domain adaptation from synthetic data to unlabeled real-world scenes. Results consistently demonstrate improved generalization in both top-performing and domain-adaptable stereo-matching models. The code for CMD will be available at: \href{https://github.com/gallenszl/CMD}{https://github.com/gallenszl/CMD}.

Figures (6)

• Figure 1: Left: Proposed unsupervised domain adaptation for stereo matching (from left to right: disparity map, error map, and uncertainty map). Red and white denote large errors and high uncertainty. Right: Disparity probability distributions for two selected pixels (red point in the picture). Pred denotes the predicted disparity in this pixel and possibility denotes the corresponding disparity probability distribution.
• Figure 2: Some toy samples of distribution sharpness evaluation. Ground truth disparity is 10px. $pred$ is the predicted disparity. The disparity searching range is from 1 to 20 with 20 hypothesis planes.
• Figure 3: Left: Function curve of $f(x) = {e^{tx}}$. Right: Corresponding disparity probability distribution when increasing $t$. P_d denotes the disparity probability distribution, and pred is the predicted disparity. Note that we select to directly change $t$ in the inference process to give an intuitive visualization in this toy sample. The $t$ is a fixed hyperparameter during training and inference in our final implementation.
• Figure 4: The ROC curves of the proposed method on KITTI2015 dataset. D1_all is used for evaluation (the lower, the better). $L_u$ denotes the proposed uncertainty minimization loss, $U_m$ is the selected uncertainty metric, and $t$ is the hyper-parameter of anisotropic soft argmin. ($t=1$, no $L_u$) denotes the baseline setting and ($t=16$, $L_u$) is our method. Density denotes the valid pixel in the predicted pseudo-label.
• Figure 5: Comparisons of cross-domain generalization on KITTI2015 trainset. The first column shows the input left images, and for each following column, the top row shows the predicted colorized disparity map and the bottom row shows the error map.