WasteGAN: Data Augmentation for Robotic Waste Sorting through Generative Adversarial Networks

TL;DR

WasteGAN tackles data scarcity in robotic waste sorting by introducing a GAN-based augmentation framework that generates image–label pairs from very small labeled sets (as few as 100 examples). By extending SemanticGAN with GenXL, dual discriminators, and a tailored loss suite (including slog activation, $cADV$, and image–label correlation loss), the approach produces more label-faithful synthetic data that improves semantic segmentation and downstream grasp planning. The method achieves measurable gains on the ZeroWaste dataset ($up to $5.8\%$) and demonstrates real-world benefits in a robotic sorting system, including improved recognition and grasp success and reduced false positives. These results indicate strong potential for deploying data-efficient, GAN-based augmentation in cluttered industrial settings and highlight avenues for faster data generation and latent-space augmentation in future work.

Abstract

Robotic waste sorting poses significant challenges in both perception and manipulation, given the extreme variability of objects that should be recognized on a cluttered conveyor belt. While deep learning has proven effective in solving complex tasks, the necessity for extensive data collection and labeling limits its applicability in real-world scenarios like waste sorting. To tackle this issue, we introduce a data augmentation method based on a novel GAN architecture called wasteGAN. The proposed method allows to increase the performance of semantic segmentation models, starting from a very limited bunch of labeled examples, such as few as 100. The key innovations of wasteGAN include a novel loss function, a novel activation function, and a larger generator block. Overall, such innovations helps the network to learn from limited number of examples and synthesize data that better mirrors real-world distributions. We then leverage the higher-quality segmentation masks predicted from models trained on the wasteGAN synthetic data to compute semantic-aware grasp poses, enabling a robotic arm to effectively recognizing contaminants and separating waste in a real-world scenario. Through comprehensive evaluation encompassing dataset-based assessments and real-world experiments, our methodology demonstrated promising potential for robotic waste sorting, yielding performance gains of up to 5.8\% in picking contaminants. The project page is available at https://github.com/bach05/wasteGAN.git

Figures (6)

• Figure 1: A general overview of the proposed pipeline for robotic waste sorting.
• Figure 2: The overall structure of the proposed wasteGAN. In particular, we highlight the usage of the newly proposed methods, i.e., the improved generator (GenXL), the image-label correlation loss ($\mathcal{L}_{imc}$), the custom adversarial loss (cADV) and the quality loss ($\mathcal{L}_q$).
• Figure 3: Results from testing the models trained on synthetic datasets generated with semanticGANli_gan_sem_seg (orange), our approach (blue) and the dataset with only 100 real samples. Black values are the performance gains of wasteGAN with respect to semanticGAN. Red values are the performance gains with respect to training on 100 samples.
• Figure 4: Frequencies of the labels in the real-world dataset (ZeroWaste zerowaste) and the generated datasets with semanticGANli_gan_sem_seg and the proposed wasteGAN. Scale is logarithmic.
• Figure 5: Our experimental setup.