TagGAN: A Generative Model for Data Tagging

TL;DR

TagGAN tackles the challenge of obtaining pixel-level disease annotations from image-level labels by learning a fine-grained disease map through dual GAN cycles that translate abnormal chest X-rays to healthy representations and back. It generates visual attribution maps and converts them into binary masks without pixel-level supervision, enabling weakly supervised, interpretable localization across multiple diseases and datasets. In evaluations on CheXpert, TBX11K, and COVID-19, TagGAN achieves strong true-positive pixel tagging (PoI ≈ 80%) and outperforms Grad-CAM and VA-GAN in localization, while preserving anatomical detail. This approach has practical implications for radiologists by reducing manual tagging workload and providing scalable, explainable disease visualization without requiring pixel-level annotations.

Abstract

Precise identification and localization of disease-specific features at the pixel-level are particularly important for early diagnosis, disease progression monitoring, and effective treatment in medical image analysis. However, conventional diagnostic AI systems lack decision transparency and cannot operate well in environments where there is a lack of pixel-level annotations. In this study, we propose a novel Generative Adversarial Networks (GANs)-based framework, TagGAN, which is tailored for weakly-supervised fine-grained disease map generation from purely image-level labeled data. TagGAN generates a pixel-level disease map during domain translation from an abnormal image to a normal representation. Later, this map is subtracted from the input abnormal image to convert it into its normal counterpart while preserving all the critical anatomical details. Our method is first to generate fine-grained disease maps to visualize disease lesions in a weekly supervised setting without requiring pixel-level annotations. This development enhances the interpretability of diagnostic AI by providing precise visualizations of disease-specific regions. It also introduces automated binary mask generation to assist radiologists. Empirical evaluations carried out on the benchmark datasets, CheXpert, TBX11K, and COVID-19, demonstrate the capability of TagGAN to outperform current top models in accurately identifying disease-specific pixels. This outcome highlights the capability of the proposed model to tag medical images, significantly reducing the workload for radiologists by eliminating the need for binary masks during training.

Figures (10)

• Figure 1: Architecture of TagGAN: (a) during the training phase, input is diseased CXR with an image-level label to an Abnormal-to-Map generator which generates a feature map, called visual attribution map, and when this map is subtracted from input abnormal image to covert it into a normal CXR pair. This generated normal pair is fed to Normal Discriminator which is trained on original healthy data. For backward cycle, this generated normal CXR is input to second generator model which converts back into abnormal and second discriminator, trained on original abnormal data, distinguishes it. (b) during testing phase, input is diseased CXR with an image-level label to Abnormal-to-Map generator which generates a feature map to visualise disease-specific features and a constraint function coverts this map into binary mask
• Figure 2: Images from datasets
• Figure 3: TBX11K with equal bounding boxes
• Figure 4: TBX11K with unequal bounding boxes
• Figure 5: Comparative results