Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Sub-Image Recapture for Multi-View 3D Reconstruction

Yanwei Wang

TL;DR

This paper tackles the memory bottleneck in learning-based multi-view stereo when reconstructing high-resolution scenes. It introduces Sub-Image Recapture (SIR), a universal strategy that splits large images into sub-images, assigns each a synthesized recapture camera, and processes them with existing SfM/MVS pipelines while preserving native resolution. The approach reduces per-step memory, enables more overlap-driven depth estimates, and improves scalability and accuracy on large imagery, demonstrated on high-resolution airborne data using COLMAP and PatchMatchNet. The work enables applying state-of-the-art MVS to arbitrarily large images and provides an open-source implementation for practitioners.

Abstract

3D reconstruction of high-resolution target remains a challenge task due to the large memory required from the large input image size. Recently developed learning based algorithms provide promising reconstruction performance than traditional ones, however, they generally require more memory than the traditional algorithms and facing scalability issue. In this paper, we developed a generic approach, sub-image recapture (SIR), to split large image into smaller sub-images and process them individually. As a result of this framework, the existing 3D reconstruction algorithms can be implemented based on sub-image recapture with significantly reduced memory and substantially improved scalability

Sub-Image Recapture for Multi-View 3D Reconstruction

TL;DR

This paper tackles the memory bottleneck in learning-based multi-view stereo when reconstructing high-resolution scenes. It introduces Sub-Image Recapture (SIR), a universal strategy that splits large images into sub-images, assigns each a synthesized recapture camera, and processes them with existing SfM/MVS pipelines while preserving native resolution. The approach reduces per-step memory, enables more overlap-driven depth estimates, and improves scalability and accuracy on large imagery, demonstrated on high-resolution airborne data using COLMAP and PatchMatchNet. The work enables applying state-of-the-art MVS to arbitrarily large images and provides an open-source implementation for practitioners.

Abstract

3D reconstruction of high-resolution target remains a challenge task due to the large memory required from the large input image size. Recently developed learning based algorithms provide promising reconstruction performance than traditional ones, however, they generally require more memory than the traditional algorithms and facing scalability issue. In this paper, we developed a generic approach, sub-image recapture (SIR), to split large image into smaller sub-images and process them individually. As a result of this framework, the existing 3D reconstruction algorithms can be implemented based on sub-image recapture with significantly reduced memory and substantially improved scalability

Paper Structure

This paper contains 11 sections, 2 equations, 4 figures.

Table of Contents

  1. INTRODUCTION
  2. SIR: AN UNIVERSAL STRATEGY
  3. SYNTHESIZED RECAPTURE CAMERA
  4. Definition of Recapture Camera
  5. SIR for SfM and MVS
  6. SIR FOR 3D RECONSTRUCTION
  7. EXPERIMENTS
  8. Datasets
  9. Implementation Details
  10. Experimental Results
  11. CONCLUSIONS

Figures (4)

  • Figure 1: Cameras associated with both the original image and one of the corresponding 4$\times$4 sub-images.
  • Figure 2: 3D reconstruction pipeline.
  • Figure 3: A tile of the 3D terrain reconstruction point cloud.
  • Figure 4: Mesh of a small area of the reconstructed 3D terrain.