AI-Assisted Diagnosis for Covid-19 CXR Screening: From Data Collection to Clinical Validation

TL;DR

This study tackles Covid-19 pneumonia detection from Chest X-ray images by building an AI-assisted diagnostic tool using a public CORDA dataset. It introduces a two-step transfer-learning pipeline, pretraining on CheXpert for radiographic findings and then classifying Covid-19 on CORDA, augmented with FairKL debiasing to mitigate site-related bias. The work includes external clinical validation by radiologists and demonstrates improvements in diagnostic accuracy and reductions in reading time when AI assistance is used. Together, these contributions offer a practical framework for deploying AI in emergency radiology and for rapid responses to future epidemics.

Abstract

In this paper, we present the major results from the Covid Radiographic imaging System based on AI (Co.R.S.A.) project, which took place in Italy. This project aims to develop a state-of-the-art AI-based system for diagnosing Covid-19 pneumonia from Chest X-ray (CXR) images. The contributions of this work are manyfold: the release of the public CORDA dataset, a deep learning pipeline for Covid-19 detection, and the clinical validation of the developed solution by expert radiologists. The proposed detection model is based on a two-step approach that, paired with state-of-the-art debiasing, provides reliable results. Most importantly, our investigation includes the actual usage of the diagnosis aid tool by radiologists, allowing us to assess the real benefits in terms of accuracy and time efficiency. Project homepage: https://corsa.di.unito.it/

Figures (3)

• Figure 1: AI-assisted diagnosis help radiologists in making more accurate and faster dagnosis. In (a) we report the average ROC curve of a pool of radiologists during external validation, in a blind setting (no AI) and in an AI-assisted setting. In (b) we compare the average diagnosis time (per image) in a blind vs. assisted setting. We notice that the average AUC increases in the AI-assisted setting, while the diagnosis is quicker on average.
• Figure 2: Example of AI report in our custom DICOM viewer. This report is shown to the evaluating radiologist alongside the CXR image. The information includes the predicted probability of Covid-19 infection and other relevant lung pathologies. For easiness of readability, probabilities $> 0.5$ are marked in red (except for the "No Finding" class).
• Figure 3: Break-out of diagnosis performance by radiologist. (a) shows the AUC score of blind and AI-assisted evaluation. While for some radiologists, especially those with higher blind scores we notice a slight decrease in AUC, the improvement is consistent in those who achieve a lower base score. On average, the AUC increases from 0.85 to 0.88, as shown in Fig.\ref{['fig:teaser']}. (b) compares the blind diagnosis time with the AI-assisted diagnosis time. The blue line represents a linear regression line, and the dashed black line represents the identity. We can see that, on average, the diagnosis time decreases as the fitted line lies below the identity for almost all radiologists.