Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

IA-MVS: Instance-Focused Adaptive Depth Sampling for Multi-View Stereo

Yinzhe Wang, Yiwen Xiao, Hu Wang, Yiping Xu, Yan Tian

TL;DR

The paper tackles depth estimation in cascaded MVS by recognizing that individual instances occupy smaller depth ranges than whole scenes. It introduces Instance-Focused Adaptive Depth Sampling (IF-ADS) to re-sample depth hypotheses per instance, coupled with a filtering mechanism based on intra-instance depth continuity priors (FIIC) to improve robustness. A conditional probabilistic model-based confidence estimation (CPC) provides reliable, stage-aware confidence measures that reflect evolving depth-hypothesis spaces. When integrated with a transformer-based baseline (MVSFormer++) and optional SAM segmentation, IA-MVS achieves state-of-the-art results on the DTU dataset and demonstrates robust, training-free applicability to existing MVS frameworks. Overall, the approach offers practical, per-instance depth refinement and probabilistic confidence modeling to enhance depth maps and 3D reconstructions in real-world pipelines.

Abstract

Multi-view stereo (MVS) models based on progressive depth hypothesis narrowing have made remarkable advancements. However, existing methods haven't fully utilized the potential that the depth coverage of individual instances is smaller than that of the entire scene, which restricts further improvements in depth estimation precision. Moreover, inevitable deviations in the initial stage accumulate as the process advances. In this paper, we propose Instance-Adaptive MVS (IA-MVS). It enhances the precision of depth estimation by narrowing the depth hypothesis range and conducting refinement on each instance. Additionally, a filtering mechanism based on intra-instance depth continuity priors is incorporated to boost robustness. Furthermore, recognizing that existing confidence estimation can degrade IA-MVS performance on point clouds. We have developed a detailed mathematical model for confidence estimation based on conditional probability. The proposed method can be widely applied in models based on MVSNet without imposing extra training burdens. Our method achieves state-of-the-art performance on the DTU benchmark. The source code is available at https://github.com/KevinWang73106/IA-MVS.

IA-MVS: Instance-Focused Adaptive Depth Sampling for Multi-View Stereo

TL;DR

The paper tackles depth estimation in cascaded MVS by recognizing that individual instances occupy smaller depth ranges than whole scenes. It introduces Instance-Focused Adaptive Depth Sampling (IF-ADS) to re-sample depth hypotheses per instance, coupled with a filtering mechanism based on intra-instance depth continuity priors (FIIC) to improve robustness. A conditional probabilistic model-based confidence estimation (CPC) provides reliable, stage-aware confidence measures that reflect evolving depth-hypothesis spaces. When integrated with a transformer-based baseline (MVSFormer++) and optional SAM segmentation, IA-MVS achieves state-of-the-art results on the DTU dataset and demonstrates robust, training-free applicability to existing MVS frameworks. Overall, the approach offers practical, per-instance depth refinement and probabilistic confidence modeling to enhance depth maps and 3D reconstructions in real-world pipelines.

Abstract

Multi-view stereo (MVS) models based on progressive depth hypothesis narrowing have made remarkable advancements. However, existing methods haven't fully utilized the potential that the depth coverage of individual instances is smaller than that of the entire scene, which restricts further improvements in depth estimation precision. Moreover, inevitable deviations in the initial stage accumulate as the process advances. In this paper, we propose Instance-Adaptive MVS (IA-MVS). It enhances the precision of depth estimation by narrowing the depth hypothesis range and conducting refinement on each instance. Additionally, a filtering mechanism based on intra-instance depth continuity priors is incorporated to boost robustness. Furthermore, recognizing that existing confidence estimation can degrade IA-MVS performance on point clouds. We have developed a detailed mathematical model for confidence estimation based on conditional probability. The proposed method can be widely applied in models based on MVSNet without imposing extra training burdens. Our method achieves state-of-the-art performance on the DTU benchmark. The source code is available at https://github.com/KevinWang73106/IA-MVS.
Paper Structure (15 sections, 8 equations, 9 figures, 2 tables)

This paper contains 15 sections, 8 equations, 9 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Learning-based MVS methods
  4. Learning-based instance segmentation methods
  5. Methodology
  6. Instance-focused adaptive depth sampling
  7. Conditional probabilistic model-Based confidence estimation
  8. Experiment
  9. Datasets and Implementation Details
  10. Experimental performance
  11. Ablation Study
  12. Instance-Focused adaptive depth sampling
  13. Filtering based on the intra-instance depth continuity priors
  14. Conditional probabilistic model-based confidence estimation
  15. Conclusion

Figures (9)

  • Figure 1: Our method's qualitative results for depth maps and quantitative results for point clouds. $I_{r}$ and GT denote reference view images and ground-truth depth maps, respectively.
  • Figure 2: Depth coverage of the entire scene and specific regions.
  • Figure 3: Containment relationships between instances.
  • Figure 4: Given reference and source images, we follow MVSFormer++ to extract features and construct initial cost volume. To enhance initial-stage depth estimation, we progressively introduce masks, perform IF-ADS and estimation, then replace corresponding regions in the prior depth map.
  • Figure 5: The initial depth estimation result for annotated instances in (a) is shown in (b). The red bounding box highlights noise, which can be mitigated in (c).
  • ...and 4 more figures