Revisiting Lesion Tracking in 3D Total Body Photography

TL;DR

This paper tackles lesion tracking in 3D TBP by integrating geometry-driven correspondence with signal-based refinement on a template mesh. It introduces a two-stage coarse-to-fine approach: first, a template-based coarse mapping via SMPL-NICP to align source/target meshes to a common template, and second, a vector-field-based refinement that transfers texture and lesion signals to the template and optimizes a tangent flow field to better align lesions. A key contribution is the first large-scale 3D TBP skin lesion tracking dataset with 25K lesion pairs across 198 subjects, enabling robust evaluation of correspondence and matching under realistic conditions, including noise in lesion detection. The method achieves state-of-the-art performance, with a 89.9% success rate at a 10 mm criterion for all annotated lesion pairs and 98.2% matching accuracy for subjects with more than 200 lesions, demonstrating robustness to texture inconsistencies, non-isometric deformations, and detection errors. Overall, the framework provides not only accurate lesion matching but also location estimates for unmatched lesions, offering practical support for clinicians in distinguishing new growth from noise in comprehensive body-wide monitoring.

Abstract

Melanoma is the most deadly form of skin cancer. Tracking the evolution of nevi and detecting new lesions across the body is essential for the early detection of melanoma. Despite prior work on longitudinal tracking of skin lesions in 3D total body photography, there are still several challenges, including 1) low accuracy for finding correct lesion pairs across scans, 2) sensitivity to noisy lesion detection, and 3) lack of large-scale datasets with numerous annotated lesion pairs. We propose a framework that takes in a pair of 3D textured meshes, matches lesions in the context of total body photography, and identifies unmatchable lesions. We start by computing correspondence maps bringing the source and target meshes to a template mesh. Using these maps to define source/target signals over the template domain, we construct a flow field aligning the mapped signals. The initial correspondence maps are then refined by advecting forward/backward along the vector field. Finally, lesion assignment is performed using the refined correspondence maps. We propose the first large-scale dataset for skin lesion tracking with 25K lesion pairs across 198 subjects. The proposed method achieves a success rate of 89.9% (at 10 mm criterion) for all pairs of annotated lesions and a matching accuracy of 98.2% for subjects with more than 200 lesions.

Figures (6)

• Figure 1: Visualization of the source and target lesions mapped to a template mesh by registering the template mesh to the source and target meshes. (a) shows the template mesh $\mathcal{M}_\mathcal{T}$. (b) shows the source mesh ($\mathcal{M}_0$) and the target mesh ($\mathcal{M}_1$). (c) shows the correspondence maps from the source/target to the template. (d) shows the source and the target lesions mapped to the template mesh. Lesions in correspondences are visualized in the same color.
• Figure 2: Visualization of the signals on a template mesh and the vector field. (a) shows the source and the target texture signals. (b) shows the source and the target lesion signals. (c) visualizes a solved vector field that explains the difference between the source and the target signals while being smooth and small.
• Figure 3: (a) The distribution of all lesion pair geodesic distances. (b) The distribution of the subject-wise geodesic distance between lesion pairs.
• Figure 4: Comparison of correspondence maps for an individual subject (subject-162) between SMPL-NICP and the proposed method. (a) shows the distribution of the geodesic distance for all the lesion pairs on the subject. (b) shows the difference in geodesic distance of each lesion pair between SMPL-NICP and the proposed method. (c) visualizes the source and the target textured meshes. (d) visualizes the source and the target texture signals brought onto the template mesh. (e) visualizes the source and the target lesions mapped to the template mesh using SMPL-NICP and the proposed method. Lesions in correspondences are in the same color.
• Figure 5: Comparison of the precision, recall, and F1 scores under different noise levels. "linear" and "quadratic" represent the methods of Ahmedt-Aristizabal et al.ahmedt2023monitoring and Zhao et al.zhao2022skin3d, respectively. The noise level is the percentage of lesions independently taken out from the source and target lesions, ranging from 5% to 30% every 5%.