A Novel Method to Improve Quality Surface Coverage in Multi-View Capture

TL;DR

This work tackles the restricted depth-of-field problem in multi-view close-range photogrammetry by formulating a total surface-coverage cost that accounts for per-point focus quality. It introduces an EM-based baseline that assigns mesh points to cameras and optimizes per-camera focus distances, and a novel k-view optimization that jointly handles small camera subsets to escape local minima. Empirical results on the 3DBodyTex dataset show that the EM and especially the 2-view approach reduce the average cost by up to 27-30% and increase the sharp, in-focus surface area by roughly 1.5–1.8k cm$^2$ across 400 meshes, demonstrating robustness and practical gains. The methods generalize to other close-range photogrammetry tasks and can extend to broader piecewise-optimizable cost functions, with 2-view emerging as a effective and computationally efficient default.

Abstract

The depth of field of a camera is a limiting factor for applications that require taking images at a short subject-to-camera distance or using a large focal length, such as total body photography, archaeology, and other close-range photogrammetry applications. Furthermore, in multi-view capture, where the target is larger than the camera's field of view, an efficient way to optimize surface coverage captured with quality remains a challenge. Given the 3D mesh of the target object and camera poses, we propose a novel method to derive a focus distance for each camera that optimizes the quality of the covered surface area. We first design an Expectation-Minimization (EM) algorithm to assign points on the mesh uniquely to cameras and then solve for a focus distance for each camera given the associated point set. We further improve the quality surface coverage by proposing a $k$-view algorithm that solves for the points assignment and focus distances by considering multiple views simultaneously. We demonstrate the effectiveness of the proposed method under various simulations for total body photography. The EM and $k$-view algorithms improve the relative cost of the baseline single-view methods by at least $24$% and $28$% respectively, corresponding to increasing the in-focus surface area by roughly $1550$ cm$^2$ and $1780$ cm$^2$. We believe the algorithms can be useful in a number of vision applications that require photogrammetric details but are limited by the depth of field.

Figures (6)

• Figure 1: The illustration of the EM and the $k$-view method. The input is the point cloud of the target $\mathcal{P}$ and cameras $\mathcal{C}$. The output is the point assignment function $\phi:\mathcal{P}\rightarrow\mathcal{C}$ and the focus distances $\mathcal{S}\in{\mathbb R}^{|\mathcal{C}|}$. The camera $c$ and its associated points $\phi^{-1}(c)$ are visualized in the same color.
• Figure 2: Visualization for the cost-determining factors and the principle of the depth of field. In (a), the orange frustum ($V^c_s\subset{\mathbb R}^3$) is clipped at the near and far depth of field (DoF) limits. The red line represents the deviation from the optical axis. In (b), $CoC$ is the circle of confusion, $s$ is the focus distance, and $D_N(s)$ and $D_F(s)$ are the near and far depth of field limits.
• Figure 3: Qualitative comparison of different methods. The first row visualizes the cost of each point. The second row shows the focus distances of cameras as black spheres. "Closest" and "Avg" are defined in \ref{['s:evaluation']}.
• Figure 4: Comparison of image quality from different methods. The first row visualizes the camera view. The second row shows the region where 2-View improves over other methods. (a) compares 2-View with "Closest". Setting the focus distance at the closest depth w.r.t. the camera does not cover the inner part of the right hand in-focus in any view, while 2-View successfully captures it with decent sharpness. (b) compares 2-View with "Avg". Similarly, using the average depth w.r.t. the camera as the focus distance does not handle body parts that are too close to the camera. (c) compares 2-View with "EM". Since 2-View mitigates local minima, we observe more body parts captured in focus using the 2-View method. We note that for regions that are out of focus in the chosen camera view using the 2-View method, they are covered in focus in other views.
• Figure 5: Evaluation of the proposed methods under different number of cameras and different camera configurations.