A Shape-Aware Total Body Photography System for In-focus Surface Coverage Optimization

TL;DR

The paper tackles the challenge of achieving high-resolution, in-focus total body photography (TBP) for skin cancer screening by introducing a shape-aware imaging pipeline. It formulates TBP as a multi-view depth-of-field optimization, solved via an EM algorithm that jointly assigns surface points to cameras and selects per-camera focus distances, adapting to diverse body shapes and poses. The approach relies on TSDF-based 3D shape estimation and a 360-degree camera setup to maximize in-focus coverage, outperforming baseline focus protocols in simulations and mannequin scans. Results show high average surface visibility and sub-0.07 mm/pixel system resolution with robust performance under calibration and pose perturbations, suggesting improved downstream lesion analysis without prohibitive scanning time. The work lays groundwork for pose-optimized TBP and multi-scan extensions, potentially enhancing automated screening pipelines.

Abstract

Total Body Photography (TBP) is becoming a useful screening tool for patients at high risk for skin cancer. While much progress has been made, existing TBP systems can be further improved for automatic detection and analysis of suspicious skin lesions, which is in part related to the resolution and sharpness of acquired images. This paper proposes a novel shape-aware TBP system automatically capturing full-body images while optimizing image quality in terms of resolution and sharpness over the body surface. The system uses depth and RGB cameras mounted on a 360-degree rotary beam, along with 3D body shape estimation and an in-focus surface optimization method to select the optimal focus distance for each camera pose. This allows for optimizing the focused coverage over the complex 3D geometry of the human body given the calibrated camera poses. We evaluate the effectiveness of the system in capturing high-fidelity body images. The proposed system achieves an average resolution of 0.068 mm/pixel and 0.0566 mm/pixel with approximately 85% and 95% of surface area in-focus, evaluated on simulation data of diverse body shapes and poses as well as a real scan of a mannequin respectively. Furthermore, the proposed shape-aware focus method outperforms existing focus protocols (e.g. auto-focus). We believe the high-fidelity imaging enabled by the proposed system will improve automated skin lesion analysis for skin cancer screening.

Figures (8)

• Figure 1: Visualization for the cost-determining factors and the principle of the depth of field. In (a), the orange frustum ($V^c_s\subset{\mathbb R}^3$) is clipped at the near and far depth of field (DoF) limits. The red line represents the deviation from the optical axis. In (b), $CoC$ is the circle of confusion, $s$ is the focus distance, and $D_N(s)$ and $D_F(s)$ are the near and far depth of field limits.
• Figure 2: Proposed pipeline. The system calibration is performed to acquire the poses of high-resolution RGB cameras and depth cameras. The subject stands at the designated position. The system captures multiple depth images by rotating 180 degree to estimate the 3D shape of the subject through TSDF-based method. The system selects a focus distance for each RGB camera pose using the proposed shape-aware focus method to optimize the quality of surface coverage. Finally, the system rotates around the subject and takes pictures.
• Figure 3: Illustration of the (a) proposed shape-aware imaging system and (b) the schematic top-view of the acquisition in a scan.
• Figure 4: Visualization of the error for 3D shape estimation for (a) SMPL-NICP and (b) TSDF method. The white-gray mesh in (a) and (b) is the estimated shape. The GT mesh is colorized based on the error so that vertices with a smaller error are colored in dark blue and vertices with a larger error are colored in light colors. The histogram visualizes the distribution of the error for each method.
• Figure 5: Visualization of (a) an example data from the 3DBodyTex dataset and (b) a set of uniformly sampled 10K points. The camera positions are visualized in black spheres, and the orientations are visualized in the red/green/blue-arrow for the x/y/z-axis.