Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Global-Local Path Networks for Monocular Depth Estimation with Vertical CutDepth

Doyeon Kim, Woonghyun Ka, Pyungwhan Ahn, Donggyu Joo, Sehwan Chun, Junmo Kim

TL;DR

This work introduces Global-Local Path Networks for monocular depth estimation, leveraging a hierarchical transformer encoder to capture global context and a compact, fusion-enabled decoder to preserve local details. A Selective Feature Fusion module enables adaptive integration of local and global features, while Vertical CutDepth provides a depth-aware augmentation that emphasizes vertical structure. The approach achieves state-of-the-art results on NYU Depth V2 with a markedly smaller decoder, and demonstrates strong cross-dataset generalization and robustness to image corruptions. Extensive ablations and appendix results on KITTI and iBims-1 corroborate the method's effectiveness and generalizability across indoor and outdoor scenes.

Abstract

Depth estimation from a single image is an important task that can be applied to various fields in computer vision, and has grown rapidly with the development of convolutional neural networks. In this paper, we propose a novel structure and training strategy for monocular depth estimation to further improve the prediction accuracy of the network. We deploy a hierarchical transformer encoder to capture and convey the global context, and design a lightweight yet powerful decoder to generate an estimated depth map while considering local connectivity. By constructing connected paths between multi-scale local features and the global decoding stream with our proposed selective feature fusion module, the network can integrate both representations and recover fine details. In addition, the proposed decoder shows better performance than the previously proposed decoders, with considerably less computational complexity. Furthermore, we improve the depth-specific augmentation method by utilizing an important observation in depth estimation to enhance the model. Our network achieves state-of-the-art performance over the challenging depth dataset NYU Depth V2. Extensive experiments have been conducted to validate and show the effectiveness of the proposed approach. Finally, our model shows better generalisation ability and robustness than other comparative models.

Global-Local Path Networks for Monocular Depth Estimation with Vertical CutDepth

TL;DR

This work introduces Global-Local Path Networks for monocular depth estimation, leveraging a hierarchical transformer encoder to capture global context and a compact, fusion-enabled decoder to preserve local details. A Selective Feature Fusion module enables adaptive integration of local and global features, while Vertical CutDepth provides a depth-aware augmentation that emphasizes vertical structure. The approach achieves state-of-the-art results on NYU Depth V2 with a markedly smaller decoder, and demonstrates strong cross-dataset generalization and robustness to image corruptions. Extensive ablations and appendix results on KITTI and iBims-1 corroborate the method's effectiveness and generalizability across indoor and outdoor scenes.

Abstract

Depth estimation from a single image is an important task that can be applied to various fields in computer vision, and has grown rapidly with the development of convolutional neural networks. In this paper, we propose a novel structure and training strategy for monocular depth estimation to further improve the prediction accuracy of the network. We deploy a hierarchical transformer encoder to capture and convey the global context, and design a lightweight yet powerful decoder to generate an estimated depth map while considering local connectivity. By constructing connected paths between multi-scale local features and the global decoding stream with our proposed selective feature fusion module, the network can integrate both representations and recover fine details. In addition, the proposed decoder shows better performance than the previously proposed decoders, with considerably less computational complexity. Furthermore, we improve the depth-specific augmentation method by utilizing an important observation in depth estimation to enhance the model. Our network achieves state-of-the-art performance over the challenging depth dataset NYU Depth V2. Extensive experiments have been conducted to validate and show the effectiveness of the proposed approach. Finally, our model shows better generalisation ability and robustness than other comparative models.
Paper Structure (21 sections, 2 equations, 5 figures, 9 tables)

This paper contains 21 sections, 2 equations, 5 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methods
  4. Global-Local Path Networks
  5. Encoder
  6. Lightweight Decoder
  7. Selective Feature Fusion
  8. Vertical CutDepth
  9. Training Loss
  10. Experiments
  11. Dataset
  12. Implementation Details
  13. Comparison with State-of-the-Arts
  14. Ablation Study
  15. Robustness of the model
  16. ...and 6 more sections

Figures (5)

  • Figure 1: Overall architecture of the proposed network. The main components of the architecture are the encoder, decoder, and skip connections with feature fusion modules.
  • Figure 2: Detailed description of the SFF module.
  • Figure 3: Qualitative comparison with previous works on the NYU Depth V2 dataset.
  • Figure 4: Examples of estimated depth maps on the SUN RGB-D dataset.
  • Figure 5: The detailed structure of (a) Baseline-DConv (b) Baseline-UNet.