Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Shifts in Doctors' Eye Movements Between Real and AI-Generated Medical Images

David C Wong, Bin Wang, Gorkem Durak, Marouane Tliba, Mohamed Amine Kerkouri, Aladine Chetouani, Ahmet Enis Cetin, Cagdas Topel, Nicolo Gennaro, Camila Vendrami, Tugce Agirlar Trabzonlu, Amir Ali Rahsepar, Laetitia Perronne, Matthew Antalek, Onural Ozturk, Gokcan Okur, Andrew C. Gordon, Ayis Pyrros, Frank H Miller, Amir A Borhani, Hatice Savas, Eric M. Hart, Elizabeth A Krupinski, Ulas Bagci

TL;DR

This study investigates how radiologists' gaze shifts when interpreting real versus AI-generated chest X-ray images. It combines real X-rays with RoentGen-generated counterparts conditioned on MIMIC-CXR reports and records eye movements from 16 radiologists performing diagnostic tasks. The analysis covers saccadic amplitudes, directions, joint distributions, temporal fixation patterns, and fixation bias maps to quantify gaze differences. Findings indicate similar initial and final attention across image types, but notable divergence in the longest and shortest fixations, suggesting different cognitive processing and scrutiny for AI-generated imagery with implications for training, model development, and regulatory guidelines.

Abstract

Eye-tracking analysis plays a vital role in medical imaging, providing key insights into how radiologists visually interpret and diagnose clinical cases. In this work, we first analyze radiologists' attention and agreement by measuring the distribution of various eye-movement patterns, including saccades direction, amplitude, and their joint distribution. These metrics help uncover patterns in attention allocation and diagnostic strategies. Furthermore, we investigate whether and how doctors' gaze behavior shifts when viewing authentic (Real) versus deep-learning-generated (Fake) images. To achieve this, we examine fixation bias maps, focusing on first, last, short, and longest fixations independently, along with detailed saccades patterns, to quantify differences in gaze distribution and visual saliency between authentic and synthetic images.

Shifts in Doctors' Eye Movements Between Real and AI-Generated Medical Images

TL;DR

This study investigates how radiologists' gaze shifts when interpreting real versus AI-generated chest X-ray images. It combines real X-rays with RoentGen-generated counterparts conditioned on MIMIC-CXR reports and records eye movements from 16 radiologists performing diagnostic tasks. The analysis covers saccadic amplitudes, directions, joint distributions, temporal fixation patterns, and fixation bias maps to quantify gaze differences. Findings indicate similar initial and final attention across image types, but notable divergence in the longest and shortest fixations, suggesting different cognitive processing and scrutiny for AI-generated imagery with implications for training, model development, and regulatory guidelines.

Abstract

Eye-tracking analysis plays a vital role in medical imaging, providing key insights into how radiologists visually interpret and diagnose clinical cases. In this work, we first analyze radiologists' attention and agreement by measuring the distribution of various eye-movement patterns, including saccades direction, amplitude, and their joint distribution. These metrics help uncover patterns in attention allocation and diagnostic strategies. Furthermore, we investigate whether and how doctors' gaze behavior shifts when viewing authentic (Real) versus deep-learning-generated (Fake) images. To achieve this, we examine fixation bias maps, focusing on first, last, short, and longest fixations independently, along with detailed saccades patterns, to quantify differences in gaze distribution and visual saliency between authentic and synthetic images.

Paper Structure

This paper contains 16 sections, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Dataset
  4. Experimental Protocol
  5. Analysis of Stimuli
  6. Fake Stimuli
  7. Real Stimuli
  8. Analysis of Population
  9. Analysis of Visual Behavior Patterns
  10. Saccadic Amplitudes
  11. Saccadic Direction
  12. Joint Distribution of Saccadic Amplitudes and Direction
  13. Temporal Analysis
  14. Spatial Analysis of Fixation of Interest
  15. Privacy and Ethics Statement
  16. ...and 1 more sections

Figures (3)

  • Figure 1: The first row displays real X-ray images, while the second row shows generated images. The diagnostic classifications of the real images are: Normal, Pleural Effusion, Atelectasis, and Hyperinflated Lungs. Black boxes are used for anonymization.
  • Figure 2: (a+b) Comparision of saccadic length distribution, (c+d) Comparision of saccadic direction distribution, (a+b) Comparision of the joint saccadic length and direction distribution.
  • Figure 3: Visualization of Bias Maps between real and fake in the order of: First, Last, Longest, Shortest.