Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Rethinking Iterative Stereo Matching from Diffusion Bridge Model Perspective

Yuguang Shi

TL;DR

The paper addresses limitations of RNN-based iterative stereo matching, where discrete updates can degrade local detail. It reframes iterative optimization as a diffusion-model task (DMIO), introducing a bridge diffusion disparity optimization, a Time-based Gated Recurrent Unit (T-GRU), and an attention-based context network with Agent Attention to preserve high-frequency information. Empirical results on Scene Flow, KITTI, and zero-shot tests demonstrate state-of-the-art or competitive performance, with 8 iterations sufficing to achieve strong results. The work advances diffusion-inspired, iterative stereo methods, offering improved edge fidelity and cross-domain robustness while acknowledging computational costs and the need for dense ground truth.

Abstract

Recently, iteration-based stereo matching has shown great potential. However, these models optimize the disparity map using RNN variants. The discrete optimization process poses a challenge of information loss, which restricts the level of detail that can be expressed in the generated disparity map. In order to address these issues, we propose a novel training approach that incorporates diffusion models into the iterative optimization process. We designed a Time-based Gated Recurrent Unit (T-GRU) to correlate temporal and disparity outputs. Unlike standard recurrent units, we employ Agent Attention to generate more expressive features. We also designed an attention-based context network to capture a large amount of contextual information. Experiments on several public benchmarks show that we have achieved competitive stereo matching performance. Our model ranks first in the Scene Flow dataset, achieving over a 7% improvement compared to competing methods, and requires only 8 iterations to achieve state-of-the-art results.

Rethinking Iterative Stereo Matching from Diffusion Bridge Model Perspective

TL;DR

The paper addresses limitations of RNN-based iterative stereo matching, where discrete updates can degrade local detail. It reframes iterative optimization as a diffusion-model task (DMIO), introducing a bridge diffusion disparity optimization, a Time-based Gated Recurrent Unit (T-GRU), and an attention-based context network with Agent Attention to preserve high-frequency information. Empirical results on Scene Flow, KITTI, and zero-shot tests demonstrate state-of-the-art or competitive performance, with 8 iterations sufficing to achieve strong results. The work advances diffusion-inspired, iterative stereo methods, offering improved edge fidelity and cross-domain robustness while acknowledging computational costs and the need for dense ground truth.

Abstract

Recently, iteration-based stereo matching has shown great potential. However, these models optimize the disparity map using RNN variants. The discrete optimization process poses a challenge of information loss, which restricts the level of detail that can be expressed in the generated disparity map. In order to address these issues, we propose a novel training approach that incorporates diffusion models into the iterative optimization process. We designed a Time-based Gated Recurrent Unit (T-GRU) to correlate temporal and disparity outputs. Unlike standard recurrent units, we employ Agent Attention to generate more expressive features. We also designed an attention-based context network to capture a large amount of contextual information. Experiments on several public benchmarks show that we have achieved competitive stereo matching performance. Our model ranks first in the Scene Flow dataset, achieving over a 7% improvement compared to competing methods, and requires only 8 iterations to achieve state-of-the-art results.
Paper Structure (15 sections, 19 equations, 9 figures, 7 tables, 1 algorithm)

This paper contains 15 sections, 19 equations, 9 figures, 7 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. RELATED WORK
  3. THE PROPOSED METHOD
  4. Feature Extractor and Cost Volume
  5. Attention-based Context Network
  6. Bridge Diffusion Disparity Optimization
  7. T-GRU-based Update Operator
  8. Loss Functions
  9. EXPERIMENTS
  10. Datasets
  11. Implementation Details
  12. Ablation Study
  13. Performance Evaluation
  14. Zero-Shot Generalization
  15. Conclusion and Future Work

Figures (9)

  • Figure 1: Row 1: Comparison of RNN's recursive multi-step prediction and the diffusion models reverse process. Row 2: Visual comparison with IGEV on middlebury dataset.
  • Figure 2: Overview of our proposed DMIO. DMIO main contribution consists of three main modules, which are attention-based context network, bridge diffusion disparity refinement, and T-GRU based update operator.
  • Figure 3: Block designs for a Channel Self-Attention, a Feed-Forward Network, and a Agent Attention.
  • Figure 4: The diffusion network architecture consists of the Gated Recurrent Unit (GRU), Time Encoder (TE), Basic Motion Encoder (BME), optional Agent Attention (AA), and Time-based Gated Recurrent Unit (T-GRU).
  • Figure 5: Infilling ground truth disparity missing values using interpolation.
  • ...and 4 more figures