Skin Lesion Analysis Toward Melanoma Detection: A Challenge at the 2017 International Symposium on Biomedical Imaging (ISBI), Hosted by the International Skin Imaging Collaboration (ISIC)
Noel C. F. Codella, David Gutman, M. Emre Celebi, Brian Helba, Michael A. Marchetti, Stephen W. Dusza, Aadi Kalloo, Konstantinos Liopyris, Nabin Mishra, Harald Kittler, Allan Halpern
TL;DR
The paper reports on the ISBI 2017 challenge, a large-scale, public benchmark for skin lesion analysis targeting melanoma detection. It evaluates three tasks—lesion segmentation, dermoscopic feature detection, and disease classification—using a substantial ISIC-derived dataset and standardized metrics. Key findings show strong gains from ensemble deep learning and external data in classification, limited participation in feature detection, and suggestions to refine evaluation metrics to better reflect clinical variability. The work underscores the value of collaborative benchmarks for advancing automated melanoma diagnosis and informs directions for future challenges and clinical integration.
Abstract
This article describes the design, implementation, and results of the latest installment of the dermoscopic image analysis benchmark challenge. The goal is to support research and development of algorithms for automated diagnosis of melanoma, the most lethal skin cancer. The challenge was divided into 3 tasks: lesion segmentation, feature detection, and disease classification. Participation involved 593 registrations, 81 pre-submissions, 46 finalized submissions (including a 4-page manuscript), and approximately 50 attendees, making this the largest standardized and comparative study in this field to date. While the official challenge duration and ranking of participants has concluded, the dataset snapshots remain available for further research and development.