Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

F-ANcGAN: An Attention-Enhanced Cycle Consistent Generative Adversarial Architecture for Synthetic Image Generation of Nanoparticles

Varun Ajith, Anindya Pal, Saumik Bhattacharya, Sayantari Ghosh

TL;DR

This paper tackles data scarcity in nanoscale SEM image analysis by introducing F-ANcGAN, an attention-enhanced cycle-consistent GAN that generates high-fidelity SEM images from segmentation masks. It integrates a Style U-Net generator with AdaIN-based style modulation and an Attention U-Net segmentation backbone, guided by a loss framework combining perceptual, pixel, and boundary-aware terms. Empirical results show a raw FID of 17.65 and post-processed FID of 10.39 on a TiO$_2$ dataset, outperforming GAN and CycleGAN baselines and exhibiting generalization to biomedical imagery, such as glioblastoma cells. The approach reduces dependence on large annotated datasets, enabling scalable synthetic data generation to improve downstream nanoparticle segmentation in resource-limited settings and potentially broader biomedical contexts.

Abstract

Nanomaterial research is becoming a vital area for energy, medicine, and materials science, and accurate analysis of the nanoparticle topology is essential to determine their properties. Unfortunately, the lack of high-quality annotated datasets drastically hinders the creation of strong segmentation models for nanoscale imaging. To alleviate this problem, we introduce F-ANcGAN, an attention-enhanced cycle consistent generative adversarial system that can be trained using a limited number of data samples and generates realistic scanning electron microscopy (SEM) images directly from segmentation maps. Our model uses a Style U-Net generator and a U-Net segmentation network equipped with self-attention to capture structural relationships and applies augmentation methods to increase the variety of the dataset. The architecture reached a raw FID score of 17.65 for TiO$_2$ dataset generation, with a further reduction in FID score to nearly 10.39 by using efficient post-processing techniques. By facilitating scalable high-fidelity synthetic dataset generation, our approach can improve the effectiveness of downstream segmentation task training, overcoming severe data shortage issues in nanoparticle analysis, thus extending its applications to resource-limited fields.

F-ANcGAN: An Attention-Enhanced Cycle Consistent Generative Adversarial Architecture for Synthetic Image Generation of Nanoparticles

TL;DR

This paper tackles data scarcity in nanoscale SEM image analysis by introducing F-ANcGAN, an attention-enhanced cycle-consistent GAN that generates high-fidelity SEM images from segmentation masks. It integrates a Style U-Net generator with AdaIN-based style modulation and an Attention U-Net segmentation backbone, guided by a loss framework combining perceptual, pixel, and boundary-aware terms. Empirical results show a raw FID of 17.65 and post-processed FID of 10.39 on a TiO dataset, outperforming GAN and CycleGAN baselines and exhibiting generalization to biomedical imagery, such as glioblastoma cells. The approach reduces dependence on large annotated datasets, enabling scalable synthetic data generation to improve downstream nanoparticle segmentation in resource-limited settings and potentially broader biomedical contexts.

Abstract

Nanomaterial research is becoming a vital area for energy, medicine, and materials science, and accurate analysis of the nanoparticle topology is essential to determine their properties. Unfortunately, the lack of high-quality annotated datasets drastically hinders the creation of strong segmentation models for nanoscale imaging. To alleviate this problem, we introduce F-ANcGAN, an attention-enhanced cycle consistent generative adversarial system that can be trained using a limited number of data samples and generates realistic scanning electron microscopy (SEM) images directly from segmentation maps. Our model uses a Style U-Net generator and a U-Net segmentation network equipped with self-attention to capture structural relationships and applies augmentation methods to increase the variety of the dataset. The architecture reached a raw FID score of 17.65 for TiO dataset generation, with a further reduction in FID score to nearly 10.39 by using efficient post-processing techniques. By facilitating scalable high-fidelity synthetic dataset generation, our approach can improve the effectiveness of downstream segmentation task training, overcoming severe data shortage issues in nanoparticle analysis, thus extending its applications to resource-limited fields.

Paper Structure

This paper contains 23 sections, 3 equations, 9 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Literature Review
  3. Methodology
  4. Model Construction
  5. CycleGAN:
  6. Style Generation:
  7. Segmentation Network:
  8. Loss Functions
  9. Generation Losses :
  10. Discriminator Loss :
  11. Segmentation Loss:
  12. Dataset and Data Pre-processing
  13. Dataset
  14. Data Pre-processing
  15. Training Configuration
  16. ...and 8 more sections

Figures (9)

  • Figure 1: Schematic of the proposed generative pipeline. The broken arrows indicate the cycle consistency between the generator and the segmentation network.
  • Figure 2: Style U-Net Architecture
  • Figure 3: Attention U-Net Architecture
  • Figure 4: Comparison of our proposed model against standard models
  • Figure 5: Raw generated image vs Post-processed generated image from $\mathrm{TiO_2}$ dataset
  • ...and 4 more figures