Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Automated Disentangling Analysis of Skin Colour for Lesion Images

Wenbo Yang, Eman Rezk, Walaa M. Moursi, Zhou Wang

TL;DR

A skin-colour disentangling framework that adapts disentanglement-by-compression to learn a structured, manipulable latent space for SCCI from unlabelled dermatology images is proposed and it is demonstrated that dataset-level augmentation and colour normalization based on this framework achieve competitive lesion classification performance.

Abstract

Machine-learning models working on skin images often have degraded performance when the skin colour captured in images (SCCI) differs between training and deployment. Such differences arise from entangled environmental factors (e.g., illumination, camera settings), and intrinsic factors (e.g., skin tone) that cannot be accurately described by a single "skin tone" scalar. To mitigate such colour mismatch, we propose a skin-colour disentangling framework that adapts disentanglement-by-compression to learn a structured, manipulable latent space for SCCI from unlabelled dermatology images. To prevent information leakage that hinders proper learning of dark colour features, we introduce a randomized, mostly monotonic decolourization mapping. To suppress unintended colour shifts of localized patterns (e.g., ink marks, scars) during colour manipulation, we further propose a geometry-aligned post-processing step. Together, these components enable faithful counterfactual editing and answering an essential question: "What would this skin condition look like under a different SCCI?", as well as direct colour transfer between images and controlled traversal along physically meaningful directions (e.g., blood perfusion, camera white balance), enabling educational visualization of skin conditions under varying SCCI. We demonstrate that dataset-level augmentation and colour normalization based on our framework achieve competitive lesion classification performance.

Automated Disentangling Analysis of Skin Colour for Lesion Images

TL;DR

A skin-colour disentangling framework that adapts disentanglement-by-compression to learn a structured, manipulable latent space for SCCI from unlabelled dermatology images is proposed and it is demonstrated that dataset-level augmentation and colour normalization based on this framework achieve competitive lesion classification performance.

Abstract

Machine-learning models working on skin images often have degraded performance when the skin colour captured in images (SCCI) differs between training and deployment. Such differences arise from entangled environmental factors (e.g., illumination, camera settings), and intrinsic factors (e.g., skin tone) that cannot be accurately described by a single "skin tone" scalar. To mitigate such colour mismatch, we propose a skin-colour disentangling framework that adapts disentanglement-by-compression to learn a structured, manipulable latent space for SCCI from unlabelled dermatology images. To prevent information leakage that hinders proper learning of dark colour features, we introduce a randomized, mostly monotonic decolourization mapping. To suppress unintended colour shifts of localized patterns (e.g., ink marks, scars) during colour manipulation, we further propose a geometry-aligned post-processing step. Together, these components enable faithful counterfactual editing and answering an essential question: "What would this skin condition look like under a different SCCI?", as well as direct colour transfer between images and controlled traversal along physically meaningful directions (e.g., blood perfusion, camera white balance), enabling educational visualization of skin conditions under varying SCCI. We demonstrate that dataset-level augmentation and colour normalization based on our framework achieve competitive lesion classification performance.
Paper Structure (14 sections, 6 equations, 4 figures, 1 table)

This paper contains 14 sections, 6 equations, 4 figures, 1 table.

Table of Contents

  1. Introduction
  2. Methodology
  3. Model Architecture for Perceived Skin Colour Modelling
  4. Decolourization
  5. Post-Processing for Colour Correction
  6. Data Augmentation and Normalization for Downstream Tasks
  7. Training Data Augmentation
  8. Colour Normalization
  9. Experiments and Results
  10. Qualitative Results of Skin Colour Matching and Sampling
  11. Exploration of the Latent Space
  12. Dataset Augmentation and Colour Normalization for Lesion Classification
  13. Ablation Study
  14. Conclusion

Figures (4)

  • Figure 1: Our model is capable of changing the captured skin colour after training.
  • Figure 2: Trajectories with physical meanings can be found in the latent space.
  • Figure 3: Architecture of the skin colour disentangling model.
  • Figure 4: Qualitative ablation study results of transferring perceived skin colours. Rows and columns are defined in the same way as in \ref{['fig:great']}(a).