Structure Unbiased Adversarial Model for Medical Image Segmentation

TL;DR

This work tackles domain shift in medical image segmentation by proposing the Structure-Unbiased Adversarial (SUA) network, which jointly mitigates structure and intensity gaps between source and target domains. SUA combines a spatial transformation block that learns forward and inverse deformations to align dominant structures with a differentiable, invertible mapping, and an intensity distribution rendering module that transfers target textures while preserving structure; the inverse deformation warps segmentation back to the original domain. Across retinal OCT, MRI-CT, and cardiac MRI tasks, SUA consistently improves segmentation performance over state-of-the-art translation methods by achieving better structure fidelity and texture alignment, as evidenced by higher metrics such as mIoU and Dice. The approach enables more reliable cross-domain segmentation and offers traceable structure correspondence, with a noted limitation for very small objects where the dominant-structure extractor may miss subtle details.

Abstract

Generative models have been widely proposed in image recognition to generate more images where the distribution is similar to that of the real ones. It often introduces a discriminator network to differentiate the real data from the generated ones. Such models utilise a discriminator network tasked with differentiating style transferred data from data contained in the target dataset. However in doing so the network focuses on discrepancies in the intensity distribution and may overlook structural differences between the datasets. In this paper we formulate a new image-to-image translation problem to ensure that the structure of the generated images is similar to that in the target dataset. We propose a simple, yet powerful Structure-Unbiased Adversarial (SUA) network which accounts for both intensity and structural differences between the training and test sets when performing image segmentation. It consists of a spatial transformation block followed by an intensity distribution rendering module. The spatial transformation block is proposed to reduce the structure gap between the two images, and also produce an inverse deformation field to warp the final segmented image back. The intensity distribution rendering module then renders the deformed structure to an image with the target intensity distribution. Experimental results show that the proposed SUA method has the capability to transfer both intensity distribution and structural content between multiple datasets.

Figures (16)

• Figure 1: Images illustrate the domain shift problem, with green and orange lines showing differences in intensity distribution and structure compared to the target image. The comparison images on the right side highlight that structural gaps, e.g. curve, position and anatomy, are challenging to mitigate through translation methods that keep the structure unchanged. Specifically, $\bm x$ represents a sample from the source dataset, while $\bm y$ represents a sample from the target dataset. The purple and green regions indicate areas where the representation in (b) differs from (c), and where the representation in (c) differs from (b), respectively.
• Figure 2: Illustrations of domain shift issues using T-SNE. Yellow and blue dots indicate images from two different domains. From left to right, figures are plotted based on: (a) original data points; (b) data points transformed by spatial deformation; (c) data points transferred by structure-preserving GAN; (d) data points transformed first by spatial deformation followed by structure-preserving GAN; and (e) data points transferred by our proposed SUA method. The second row shows some examples.
• Figure 3: Illustration of pipelines of translation methods for segmentation: (1) Trained segmentation model without domain adaptation; (2) Translation GAN; (3) Structure preserving GANs, and (4) our GAN with learnable deformation (more details in Fig. \ref{['pipline']}).
• Figure 4: An illustration of the SUA: Firstly, the source and target images ($\bm x^{(i)}_{S}$ and $\bm x^{(i)}_{T}$) are processed to compute the dominant structure of the input image. Then, the obtained dominant structures ${\bm u_{S}}^{(i)}$ and ${\bm u_{T}}^{(i)}$ are used to compute the deformation field $\bm \phi^{(i)}$ and its inverse $\bm \phi^{-1(i)}$. The deformation field $\bm \phi^{(i)}$ is used to warp the ${\bm u_{S}}^{(i)}$, which is further processed by the generator $G$. The resultant image is fed to the trained segmentation model, whose output is warped back by the inverse deformation field $\bm \phi^{-1(i)}$ to get the final segmentation result.
• Figure 5: Intermediate results produced by the pipeline. From left to right are target image $\bm x_T$, source image $\bm x_S$, the forward deformation field $\phi$, the warped top-structure, the rendered top-structure, the prediction results, the inverse deformation field $\phi^{-1}$, and the prediction results warped back to the original structure.