Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

From Majority to Minority: A Diffusion-based Augmentation for Underrepresented Groups in Skin Lesion Analysis

Janet Wang, Yunsung Chung, Zhengming Ding, Jihun Hamm

TL;DR

Problem: Skin lesion classifiers often underperform on minority skin types due to limited representation in training data. Approach: Introduce a diffusion-based augmentation framework that leverages majority-group lesion information using Latent Diffusion Models, Textual Inversion for concept discovery, and LoRA for fine-grained specialization. Experiments: Evaluate on the Fitzpatrick17k dataset across three data-scarcity scenarios and multiple backbones (VGG-16, ResNet-18, ViT-B-16). Findings: Synthetic data generated by the framework improves minority-group diagnosis, including scenarios with little or no minority data, and the best gains arise from combining real and synthetic data along with dual-guided generation. Significance: Demonstrates practical pathways to reduce under-diagnosis in medical imaging by reusing rich data from majority groups and leveraging pretrained diffusion knowledge.

Abstract

AI-based diagnoses have demonstrated dermatologist-level performance in classifying skin cancer. However, such systems are prone to under-performing when tested on data from minority groups that lack sufficient representation in the training sets. Although data collection and annotation offer the best means for promoting minority groups, these processes are costly and time-consuming. Prior works have suggested that data from majority groups may serve as a valuable information source to supplement the training of diagnosis tools for minority groups. In this work, we propose an effective diffusion-based augmentation framework that maximizes the use of rich information from majority groups to benefit minority groups. Using groups with different skin types as a case study, our results show that the proposed framework can generate synthetic images that improve diagnostic results for the minority groups, even when there is little or no reference data from these target groups. The practical value of our work is evident in medical imaging analysis, where under-diagnosis persists as a problem for certain groups due to insufficient representation.

From Majority to Minority: A Diffusion-based Augmentation for Underrepresented Groups in Skin Lesion Analysis

TL;DR

Problem: Skin lesion classifiers often underperform on minority skin types due to limited representation in training data. Approach: Introduce a diffusion-based augmentation framework that leverages majority-group lesion information using Latent Diffusion Models, Textual Inversion for concept discovery, and LoRA for fine-grained specialization. Experiments: Evaluate on the Fitzpatrick17k dataset across three data-scarcity scenarios and multiple backbones (VGG-16, ResNet-18, ViT-B-16). Findings: Synthetic data generated by the framework improves minority-group diagnosis, including scenarios with little or no minority data, and the best gains arise from combining real and synthetic data along with dual-guided generation. Significance: Demonstrates practical pathways to reduce under-diagnosis in medical imaging by reusing rich data from majority groups and leveraging pretrained diffusion knowledge.

Abstract

AI-based diagnoses have demonstrated dermatologist-level performance in classifying skin cancer. However, such systems are prone to under-performing when tested on data from minority groups that lack sufficient representation in the training sets. Although data collection and annotation offer the best means for promoting minority groups, these processes are costly and time-consuming. Prior works have suggested that data from majority groups may serve as a valuable information source to supplement the training of diagnosis tools for minority groups. In this work, we propose an effective diffusion-based augmentation framework that maximizes the use of rich information from majority groups to benefit minority groups. Using groups with different skin types as a case study, our results show that the proposed framework can generate synthetic images that improve diagnostic results for the minority groups, even when there is little or no reference data from these target groups. The practical value of our work is evident in medical imaging analysis, where under-diagnosis persists as a problem for certain groups due to insufficient representation.

Paper Structure

This paper contains 9 sections, 3 equations, 2 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Methods
  3. Latent Diffusion Models
  4. Concept Discovery via Textual Inversion
  5. Fine-grained Detail Enhancement with LoRA
  6. Experiments
  7. Results
  8. Conclusion
  9. Acknowledgments.

Figures (2)

  • Figure 1: Overview of the proposed augmentation framework. The framework pairs each training image with a textual prompt describing the condition as an input to train a latent diffusion model. Embeddings associated with new lesion concepts are found through Textual Inversion. Compact matrices $A$ and $B$ are optimized via LoRA to facilitate training with the new embeddings. During inference, the trained model produces synthetic images from the training set that mainly features the majority groups via image-to-image generation, thus conditioned on visual cues of lesions from images and textual prompts describing the target condition and group attributes.
  • Figure 2: Examples of synthetic images generated by a model trained exclusively on light-skinned images, using prompts describing dark skin types.