Monitoring of Pigmented Skin Lesions Using 3D Whole Body Imaging

TL;DR

3D whole-body imaging system known as 3DSkin-mapper can be used by dermatological clinics to conduct skin screening, detect and track skin lesions over time, identify suspicious lesions, and document pigmented lesions.

Abstract

Advanced artificial intelligence and machine learning have great potential to redefine how skin lesions are detected, mapped, tracked and documented. Here, We propose a 3D whole-body imaging system known as 3DSkin-mapper to enable automated detection, evaluation and mapping of skin lesions. A modular camera rig arranged in a cylindrical configuration was designed to automatically capture images of the entire skin surface of a subject synchronously from multiple angles. Based on the images, we developed algorithms for 3D model reconstruction, data processing and skin lesion detection and tracking based on deep convolutional neural networks. We also introduced a customised, user-friendly, and adaptable interface that enables individuals to interactively visualise, manipulate, and annotate the images. The proposed system is developed for skin lesion screening, the focus of this paper is to introduce the system instead of clinical study. Using synthetic and real images we demonstrate the effectiveness of the proposed system by providing multiple views of a target skin lesion, enabling further 3D geometry analysis and longitudinal tracking. It takes only a few seconds to capture the entire skin surface, and about half an hour to process and analyse the images. Our experiments show that the proposed system allow fast and easy whole body 3D imaging. It can be used by dermatological clinics to conduct skin screening, detect and track skin lesions over time, identify suspicious lesions, and document pigmented lesions. The system can potentially save clinicians time and effort significantly. The 3D imaging and analysis has the potential to change the paradigm of whole body photography with many applications in skin diseases, including inflammatory and pigmentary disorders.

Figures (19)

• Figure 1: 3D Whole body imaging captures the entire skin surface to map, document and monitor pigmented lesions. Visualisation of the sparse point cloud (left), 3D texture model (middle), and skin lesions detected (right). All lesions detected on 2D images are mapped to the 3D reconstructed model, improving the ability to track lesions over time. Cameras are shown as 3D wireframe pyramids.
• Figure 2: Overview of the 3DSkin-mapper workflow. 1. Design and construction of the hardware and software for the data collection (60 cameras). 2. Collection of diverse human data for the 3D reconstruction, 3D fine-tuning and skin lesion detection. 3. Reconstruction of the human body from different camera poses and sparse point cloud estimation. 4. Estimation of the depth, human mask and 2D to 3D projection. 5. A deep convolutional neural network that detects lesions from given wide-field images. 6. 3DSkin-mapper user interface for visualisation, curation and data management.
• Figure 3: Whole body photography structure. Camera cage which consist of 15 poles arranged in a cylindrical configuration, each pole with 4 cameras Canon EOS 200D.
• Figure 4: Top: Block diagram of the camera triggering circuit. Bottom: Screenshot of the camera control application running on Windows OS.
• Figure 5: Sample images collected from a participant with the proposed whole body imaging system. Selected poles from the imaging system are shown as orange circles.