FairDiff: Fair Segmentation with Point-Image Diffusion

TL;DR

This work tackles fairness in medical image segmentation under data imbalance by introducing a data-driven augmentation approach that synthesizes labeled images for underrepresented groups. It proposes Point-Image Diffusion, a two-stage pipeline that converts 2D segmentation masks into 3D point clouds for boundary-aware diffusion, followed by a ControlNet-conditioned image synthesis stage, and combines real and synthetic data with Equal-Scale balancing. The method demonstrates superior image-synthesis quality (lower $\text{FID}$ and favorable $\text{MMD}$) and improved fairness-aware segmentation (higher ES-Dice and ES-IoU across multiple attributes) compared with state-of-the-art baselines. By enabling balanced training data and leveraging joint optimization, the approach enhances equitable medical segmentation and offers a scalable path toward reducing disparities in clinical imaging, with code available at the referenced repository.

Abstract

Fairness is an important topic for medical image analysis, driven by the challenge of unbalanced training data among diverse target groups and the societal demand for equitable medical quality. In response to this issue, our research adopts a data-driven strategy-enhancing data balance by integrating synthetic images. However, in terms of generating synthetic images, previous works either lack paired labels or fail to precisely control the boundaries of synthetic images to be aligned with those labels. To address this, we formulate the problem in a joint optimization manner, in which three networks are optimized towards the goal of empirical risk minimization and fairness maximization. On the implementation side, our solution features an innovative Point-Image Diffusion architecture, which leverages 3D point clouds for improved control over mask boundaries through a point-mask-image synthesis pipeline. This method outperforms significantly existing techniques in synthesizing scanning laser ophthalmoscopy (SLO) fundus images. By combining synthetic data with real data during the training phase using a proposed Equal Scale approach, our model achieves superior fairness segmentation performance compared to the state-of-the-art fairness learning models. Code is available at https://github.com/wenyi-li/FairDiff.

Figures (3)

• Figure 1: Comparison of Traditional Noise-Image Diffusion and Our Point-Image Diffusion Methods. We transform 2D mask data into a 3D point cloud format, leveraging the spatial coordinates to delineate boundaries more accurately.
• Figure 2: Overview of Our Fairness-aware Point-Image Diffusion Framework.
• Figure 3: Visualization Results of Different Image Synthesis Results.