Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Automated Measurement of Eczema Severity with Self-Supervised Learning

Neelesh Kumar, Oya Aran

TL;DR

The paper tackles automated eczema severity assessment from in-the-wild images under limited labeled data. It introduces a two-stage approach: SegGPT-based few-shot segmentation to isolate eczema regions, followed by DINO self-supervised feature extraction from the segmented region and MLP-based severity classification. On an in-the-wild eczema dataset, the method achieves a weighted F1 of $0.67 ± 0.01$, outperforming finetuned ResNet-18 ($0.44 ± 0.16$) and ViT-B ($0.40 ± 0.22$), demonstrating the effectiveness of self-supervised representations and in-context segmentation in data-scarce skin-disease settings. The results suggest strong potential for data-efficient, real-world dermatology tools and point to future enhancements via domain-specific fine-tuning and synthetic data augmentation.

Abstract

Automated diagnosis of eczema using images acquired from digital camera can enable individuals to self-monitor their recovery. The process entails first segmenting out the eczema region from the image and then measuring the severity of eczema in the segmented region. The state-of-the-art methods for automated eczema diagnosis rely on deep neural networks such as convolutional neural network (CNN) and have shown impressive performance in accurately measuring the severity of eczema. However, these methods require massive volume of annotated data to train which can be hard to obtain. In this paper, we propose a self-supervised learning framework for automated eczema diagnosis under limited training data regime. Our framework consists of two stages: i) Segmentation, where we use an in-context learning based algorithm called SegGPT for few-shot segmentation of eczema region from the image; ii) Feature extraction and classification, where we extract DINO features from the segmented regions and feed it to a multi-layered perceptron (MLP) for 4-class classification of eczema severity. When evaluated on a dataset of annotated "in-the-wild" eczema images, we show that our method outperforms (Weighted F1: 0.67 $\pm$ 0.01) the state-of-the-art deep learning methods such as finetuned Resnet-18 (Weighted F1: 0.44 $\pm$ 0.16) and Vision Transformer (Weighted F1: 0.40 $\pm$ 0.22). Our results show that self-supervised learning can be a viable solution for automated skin diagnosis where labeled data is scarce.

Automated Measurement of Eczema Severity with Self-Supervised Learning

TL;DR

The paper tackles automated eczema severity assessment from in-the-wild images under limited labeled data. It introduces a two-stage approach: SegGPT-based few-shot segmentation to isolate eczema regions, followed by DINO self-supervised feature extraction from the segmented region and MLP-based severity classification. On an in-the-wild eczema dataset, the method achieves a weighted F1 of , outperforming finetuned ResNet-18 () and ViT-B (), demonstrating the effectiveness of self-supervised representations and in-context segmentation in data-scarce skin-disease settings. The results suggest strong potential for data-efficient, real-world dermatology tools and point to future enhancements via domain-specific fine-tuning and synthetic data augmentation.

Abstract

Automated diagnosis of eczema using images acquired from digital camera can enable individuals to self-monitor their recovery. The process entails first segmenting out the eczema region from the image and then measuring the severity of eczema in the segmented region. The state-of-the-art methods for automated eczema diagnosis rely on deep neural networks such as convolutional neural network (CNN) and have shown impressive performance in accurately measuring the severity of eczema. However, these methods require massive volume of annotated data to train which can be hard to obtain. In this paper, we propose a self-supervised learning framework for automated eczema diagnosis under limited training data regime. Our framework consists of two stages: i) Segmentation, where we use an in-context learning based algorithm called SegGPT for few-shot segmentation of eczema region from the image; ii) Feature extraction and classification, where we extract DINO features from the segmented regions and feed it to a multi-layered perceptron (MLP) for 4-class classification of eczema severity. When evaluated on a dataset of annotated "in-the-wild" eczema images, we show that our method outperforms (Weighted F1: 0.67 0.01) the state-of-the-art deep learning methods such as finetuned Resnet-18 (Weighted F1: 0.44 0.16) and Vision Transformer (Weighted F1: 0.40 0.22). Our results show that self-supervised learning can be a viable solution for automated skin diagnosis where labeled data is scarce.

Paper Structure

This paper contains 11 sections, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Methods
  3. SegGPT for Eczema Segmentation
  4. Self-Supervised Learning with DINO features
  5. Experiments and Results
  6. Dataset description
  7. Baselines for comparison
  8. Automated Eczema Diagnosis with the Proposed Framework
  9. Ablation Study
  10. Limited Training Data Regime
  11. Discussion

Figures (2)

  • Figure 1: Overview. Our framework consists of two stages: In the first stage, we use a pretrained Seg-GPT model to perform segmentation of eczema region using just 2 labeled examples in the prompt. In the second stage, we use a pretrained DINO model to extract features from the segmented region. The features are then used to train an MLP to perform classification of eczema severity levels.
  • Figure 2: Training with limited data. Shown here are the means and standard deviations (red bars) for weighted averaged F1 score at different percentages of training data used. Even when trained with just 20% of training data, the method exhibits only a slight decrease in performance.