PoTATO: A Dataset for Analyzing Polarimetric Traces of Afloat Trash Objects

TL;DR

PoTATO tackles robust on-water object detection under challenging outdoor illumination by introducing a first-of-its-kind dataset with raw polarimetric and chromatic data for litter detection. The dataset (12,380 bottle annotations across six modalities: $MONO$, $RGB$, $DIF$, $DOLP$, $POL$, $PAULI$) is accompanied by a processing pipeline to compute polarimetric metrics from a microgrid polarimeter, enabling six visualization channels and multi-modal fusion experiments. Across three detectors, fine-tuned models benefit from polarimetric inputs, with $DIF$ and $POL$ often surpassing RGB for medium/large objects, while RGB remains superior for small ones; these findings highlight the complementary strengths of polarization under real-world water-surface conditions. The work provides public data and code to spur research on fusion of color and polarization and to advance robust perception in mobile robotics addressing marine litter.

Abstract

Plastic waste in aquatic environments poses severe risks to marine life and human health. Autonomous robots can be utilized to collect floating waste, but they require accurate object identification capability. While deep learning has been widely used as a powerful tool for this task, its performance is significantly limited by outdoor light conditions and water surface reflection. Light polarization, abundant in such environments yet invisible to the human eye, can be captured by modern sensors to significantly improve litter detection accuracy on water surfaces. With this goal in mind, we introduce PoTATO, a dataset containing 12,380 labeled plastic bottles and rich polarimetric information. We demonstrate under which conditions polarization can enhance object detection and, by providing raw image data, we offer an opportunity for the research community to explore novel approaches and push the boundaries of state-of-the-art object detection algorithms even further. Code and data are publicly available at https://github.com/luisfelipewb/ PoTATO/tree/eccv2024.

Figures (8)

• Figure 1: Different modalities that can be extracted from the dataset: Grayscale (MONO), Color (RGB), Color image with diffuse-only reflections (DIF), Degree of Linear Polarization (DOLP), pseudo-color image combining degree and angle of polarization (POL), and pseudo-color Pauli-inspired image (PAULI)
• Figure 2: Polarized camera mounted on the Kingfisher USV
• Figure 3: Size distribution of bounding boxes.
• Figure 4: Main natural sources of light polarization
• Figure 5: Super Pixel, Extraction Pipeline and Color Encoding