Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Modeling Stereo-Confidence Out of the End-to-End Stereo-Matching Network via Disparity Plane Sweep

Jae Young Lee, Woonghyun Ka, Jaehyun Choi, Junmo Kim

TL;DR

The paper tackles the challenge of obtaining reliable stereo-confidence without accessing internal cost volumes of end-to-end stereo-matching networks, which is important for safety-critical systems. It introduces a disparity-profile-based confidence measure derived from disparity plane sweep, constructing a disparity volume and comparing an ideal linear profile against observed profiles anchored at zero shift. The method shows competitive performance with conventional confidence approaches and enhances learning-based confidence when used as an extra input modality, demonstrating versatility across multiple datasets and networks. The approach offers a practical external cue that can augment or substitute cost-volume information, with potential for self-supervised disparity refinement in the future.

Abstract

We propose a novel stereo-confidence that can be measured externally to various stereo-matching networks, offering an alternative input modality choice of the cost volume for learning-based approaches, especially in safety-critical systems. Grounded in the foundational concepts of disparity definition and the disparity plane sweep, the proposed stereo-confidence method is built upon the idea that any shift in a stereo-image pair should be updated in a corresponding amount shift in the disparity map. Based on this idea, the proposed stereo-confidence method can be summarized in three folds. 1) Using the disparity plane sweep, multiple disparity maps can be obtained and treated as a 3-D volume (predicted disparity volume), like the cost volume is constructed. 2) One of these disparity maps serves as an anchor, allowing us to define a desirable (or ideal) disparity profile at every spatial point. 3) By comparing the desirable and predicted disparity profiles, we can quantify the level of matching ambiguity between left and right images for confidence measurement. Extensive experimental results using various stereo-matching networks and datasets demonstrate that the proposed stereo-confidence method not only shows competitive performance on its own but also consistent performance improvements when it is used as an input modality for learning-based stereo-confidence methods.

Modeling Stereo-Confidence Out of the End-to-End Stereo-Matching Network via Disparity Plane Sweep

TL;DR

The paper tackles the challenge of obtaining reliable stereo-confidence without accessing internal cost volumes of end-to-end stereo-matching networks, which is important for safety-critical systems. It introduces a disparity-profile-based confidence measure derived from disparity plane sweep, constructing a disparity volume and comparing an ideal linear profile against observed profiles anchored at zero shift. The method shows competitive performance with conventional confidence approaches and enhances learning-based confidence when used as an extra input modality, demonstrating versatility across multiple datasets and networks. The approach offers a practical external cue that can augment or substitute cost-volume information, with potential for self-supervised disparity refinement in the future.

Abstract

We propose a novel stereo-confidence that can be measured externally to various stereo-matching networks, offering an alternative input modality choice of the cost volume for learning-based approaches, especially in safety-critical systems. Grounded in the foundational concepts of disparity definition and the disparity plane sweep, the proposed stereo-confidence method is built upon the idea that any shift in a stereo-image pair should be updated in a corresponding amount shift in the disparity map. Based on this idea, the proposed stereo-confidence method can be summarized in three folds. 1) Using the disparity plane sweep, multiple disparity maps can be obtained and treated as a 3-D volume (predicted disparity volume), like the cost volume is constructed. 2) One of these disparity maps serves as an anchor, allowing us to define a desirable (or ideal) disparity profile at every spatial point. 3) By comparing the desirable and predicted disparity profiles, we can quantify the level of matching ambiguity between left and right images for confidence measurement. Extensive experimental results using various stereo-matching networks and datasets demonstrate that the proposed stereo-confidence method not only shows competitive performance on its own but also consistent performance improvements when it is used as an input modality for learning-based stereo-confidence methods.
Paper Structure (18 sections, 6 equations, 3 figures, 6 tables)

This paper contains 18 sections, 6 equations, 3 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Conventional Methods
  4. Learning-Based Methods
  5. Method
  6. Preliminaries and Concept
  7. Reinterpretation of Confidence
  8. Obtaining Disparity Profiles
  9. Proposed Stereo-Confidence Method
  10. Experiments
  11. Datasets
  12. Stereo-Matching Networks
  13. Confidence Networks
  14. Evaluation Metrics
  15. Confidence Estimation Analysis
  16. ...and 3 more sections

Figures (3)

  • Figure 1: Conceptual description of ideal and non-ideal profiles according to the disparity plane sweep in signal processing perspective.
  • Figure 2: The entire process of the proposed method for quantifying unreliability (i.e., matching ambiguity) and measuring confidence out of the stereo-matching network using disparity plane sweep and the observations of disparity profiles sampled from corresponding pixels in the left image $I_{L}$.
  • Figure 3: The confidence maps on K2012 ($1^{st}$ and $2^{nd}$ rows), K2015 ($3^{rd}$ and $4^{th}$ rows), VK2-S6 ($5^{th}$ and $6^{th}$ rows), and M2014 (last two rows) datasets using PSMNet. (From top to bottom, left to right) left image, predicted disparity map, estimated confidence maps by Ours, CCNN, CCNN$^{\dagger}$, LFN, ConfNet, LGC, LAF-Net$^*$, LAF-Net$^{*\dagger}$, LAF-Net, and LAF-Net$^{\dagger}$.