Hybrid Vision Transformer_GAN Attribute Neutralizer for Mitigating Bias in Chest X_Ray Diagnosis

TL;DR

This study tests whether swapping the CNN-based U‑Net encoder in an Attribute-Neutral Framework for chest X-ray analysis with a Vision Transformer (DeiT-Small) can further suppress demographic attribute leakage (sex and age) without harming diagnostic accuracy. At moderate edit strength, the ViT neutralizer reduces leakage to about 0.80 AUC for sex and 0.80 AUC for age, while preserving roughly 95–97% of baseline macro diagnostic ROC–AUC across 15 findings, and keeping worst-case subgroup performance near 0.70. The AI judge leakage drops dramatically at stronger edits (AUC ≤ 0.10 for α ≥ 0.6), demonstrating effective neutralization of demographic cues, with Grad-CAM analyses supporting changes in attribute-bearing regions. These results suggest that global self-attention can enhance pixel-space debiasing in medical imaging, offering a practical path toward fairer chest X‑ray AI, though validation across more datasets and attributes remains needed.

Abstract

Bias in chest X-ray classifiers frequently stems from sex- and age-related shortcuts, leading to systematic underdiagnosis of minority subgroups. Previous pixel-space attribute neutralizers, which rely on convolutional encoders, lessen but do not fully remove this attribute leakage at clinically usable edit strengths. This study evaluates whether substituting the U-Net convolutional encoder with a Vision Transformer backbone in the Attribute-Neutral Framework can reduce demographic attribute leakage while preserving diagnostic accuracy. A data-efficient Image Transformer Small (DeiT-S) neutralizer was trained on the ChestX-ray14 dataset. Its edited images, generated across eleven edit-intensity levels, were evaluated with an independent AI judge for attribute leakage and with a convolutional neural network (ConvNet) for disease prediction. At a moderate edit level (alpha = 0.5), the Vision Transformer (ViT) neutralizer reduces patient sex-recognition area under the curve (AUC) to approximately 0.80, about 10 percentage points below the original framework's convolutional U-Net encoder, despite being trained for only half as many epochs. Meanwhile, macro receiver operating characteristic area under the curve (ROC AUC) across 15 findings stays within five percentage points of the unedited baseline, and the worst-case subgroup AUC remains near 0.70. These results indicate that global self-attention vision models can further suppress attribute leakage without sacrificing clinical utility, suggesting a practical route toward fairer chest X-ray AI.

Figures (38)

• Figure 1: Joint distribution of sex and age bins. Young males dominate, whereas females aged $\ge 60$ form the smallest subgroup (Wilson 95 % CI for their proportion excludes 10 %), motivating the up-weighting strategy used during balanced sampling.
• Figure 2: Frequencies of the 15 diagnostic labels in ChestX-ray14. The most common class (“No Finding’’) is over three orders of magnitude more frequent than the rarest (“Hernia’’), illustrating the pronounced long-tailed label distribution that the loss re-weighting scheme must address.
• Figure 3: End-to-end pipeline: ChestX-ray14 images are pre-processed, edited by two neutralizers at 11 $\alpha$ levels, and then (i) scored for attribute leakage by an AI-Judge and (ii) used to train a Disease-Diagnosis Model (DDM). The DDM trained on original images plus two alternative debiasing baselines serve as comparators. All models are evaluated on a common test set for accuracy and fairness.
• Figure 4: The Attribute Neutralizer training and image-generation processhu2024enhancing.
• Figure 5: ViT-adapted generator. A learnable 1 × 1 convolution maps the single-channel X-ray to Red, Green, and Blue (RGB). A DeiT-Small transformer encoder (12 blocks, 16 × 16 patches, 384-dim) replaces the original CNN encoder, skip connections are removed, and the decoder depth is reduced to four up-convolution stages.