The Impact of AI Assistance on Radiology Reporting: A Pilot Study Using Simulated AI Draft Reports
Julián N. Acosta, Siddhant Dogra, Subathra Adithan, Kay Wu, Michael Moritz, Stephen Kwak, Pranav Rajpurkar
TL;DR
The study addresses radiology reporting workload by testing AI-generated draft reports as a starting point in a chest CT workflow. Using a three-reader, multi-case crossover design with GPT-4-simulated drafts (including deliberate errors), it finds a 24% reduction in median reporting time without a significant drop in diagnostic accuracy. The results suggest AI-assisted drafting can improve efficiency while maintaining safety, though findings are tempered by small sample size and simulated outputs. If replicated in larger trials with real AI drafts, this approach could meaningfully alleviate clinician workload and enhance reporting throughput in clinical practice.
Abstract
Radiologists face increasing workload pressures amid growing imaging volumes, creating risks of burnout and delayed reporting times. While artificial intelligence (AI) based automated radiology report generation shows promise for reporting workflow optimization, evidence of its real-world impact on clinical accuracy and efficiency remains limited. This study evaluated the effect of draft reports on radiology reporting workflows by conducting a three reader multi-case study comparing standard versus AI-assisted reporting workflows. In both workflows, radiologists reviewed the cases and modified either a standard template (standard workflow) or an AI-generated draft report (AI-assisted workflow) to create the final report. For controlled evaluation, we used GPT-4 to generate simulated AI drafts and deliberately introduced 1-3 errors in half the cases to mimic real AI system performance. The AI-assisted workflow significantly reduced average reporting time from 573 to 435 seconds (p=0.003), without a statistically significant difference in clinically significant errors between workflows. These findings suggest that AI-generated drafts can meaningfully accelerate radiology reporting while maintaining diagnostic accuracy, offering a practical solution to address mounting workload challenges in clinical practice.