Self-Supervised Learning with Generative Adversarial Networks for Electron Microscopy
Bashir Kazimi, Karina Ruzaeva, Stefan Sandfeld
TL;DR
The paper addresses limited annotated data in electron microscopy by applying self-supervised pretraining with a conditional GAN on unlabeled data (CEM500K), followed by fine-tuning for downstream tasks such as segmentation, denoising, background removal, and super-resolution. It uses a Pix2Pix-style objective with $L_{cGAN}$ and a through $L_{1}$ loss weighted by $\lambda$ to guide learning, enabling faster convergence and better generalization across architectures and receptive fields. The key findings show that smaller pretrained networks can match or exceed larger randomly initialized models, and HRNet-based pretrained models often achieve the best performance on Au nanoparticle segmentation and TEMImageNet tasks. This approach reduces annotation costs and hyperparameter tuning while enabling scalable EM analysis under limited labeled data.
Abstract
In this work, we explore the potential of self-supervised learning with Generative Adversarial Networks (GANs) for electron microscopy datasets. We show how self-supervised pretraining facilitates efficient fine-tuning for a spectrum of downstream tasks, including semantic segmentation, denoising, noise \& background removal, and super-resolution. Experimentation with varying model complexities and receptive field sizes reveals the remarkable phenomenon that fine-tuned models of lower complexity consistently outperform more complex models with random weight initialization. We demonstrate the versatility of self-supervised pretraining across various downstream tasks in the context of electron microscopy, allowing faster convergence and better performance. We conclude that self-supervised pretraining serves as a powerful catalyst, being especially advantageous when limited annotated data are available and efficient scaling of computational cost is important.