The TYC Dataset for Understanding Instance-Level Semantics and Motions of Cells in Microstructures

TL;DR

The paper introduces the trapped yeast cell (TYC) dataset, a large-scale resource of high-resolution brightfield microscopy images and unlabeled video clips designed to study instance-level cell semantics and motions within microstructured traps. It provides 105 densely annotated images with cell and trap masks and 261 unlabeled clips to support supervised and unsupervised learning, respectively, across diverse trap geometries. A standardized evaluation framework using IoU for semantic-level assessment and Panoptic Quality (PQ) for instance-level assessment enables fair cross-study comparisons, with accompanying PyTorch code. Qualitative results with SAM illustrate the dataset’s complexity—especially in scenarios with touching cells or debris—highlighting the need for dedicated segmentation and tracking approaches in microstructured environments. Overall, TYC aims to advance biomedical image analysis by enabling robust cell segmentation and motion understanding in microfluidic contexts and by providing a public, CC BY 4.0-licensed benchmark for reproducibility and cross-lab collaboration.

Abstract

Segmenting cells and tracking their motion over time is a common task in biomedical applications. However, predicting accurate instance-wise segmentation and cell motions from microscopy imagery remains a challenging task. Using microstructured environments for analyzing single cells in a constant flow of media adds additional complexity. While large-scale labeled microscopy datasets are available, we are not aware of any large-scale dataset, including both cells and microstructures. In this paper, we introduce the trapped yeast cell (TYC) dataset, a novel dataset for understanding instance-level semantics and motions of cells in microstructures. We release $105$ dense annotated high-resolution brightfield microscopy images, including about $19$k instance masks. We also release $261$ curated video clips composed of $1293$ high-resolution microscopy images to facilitate unsupervised understanding of cell motions and morphology. TYC offers ten times more instance annotations than the previously largest dataset, including cells and microstructures. Our effort also exceeds previous attempts in terms of microstructure variability, resolution, complexity, and capturing device (microscopy) variability. We facilitate a unified comparison on our novel dataset by introducing a standardized evaluation strategy. TYC and evaluation code are publicly available under CC BY 4.0 license.

Figures (18)

• Figure 1: Dataset teaser. Samples of our trapped yeast cell dataset of high-resolution ($\geq2048\times2048$) brightfield images overlaid with multi-class instance segmentation labels; shades of (dark) grey ( $\blacksquare\!\!\!\!\blacksquare$$\blacksquare\!\!\!\!\blacksquare$$\blacksquare\!\!\!\!\blacksquare$) indicate microstructures (traps) and shades of pink ( $\blacksquare\!\!\!\!\blacksquare$$\blacksquare\!\!\!\!\blacksquare$$\blacksquare\!\!\!\!\blacksquare$) indicate individual cell instances. Image brightness adjusted for better visualization. Best viewed in color; zoom in for details.
• Figure 2: Overview data acquisition process. General TLFM experiment setup for single-cell fluorescence measurement. Microscope (from left to right), microfluidic chip (cell chamber within the purple dot), trap chamber (pink rectangle), trap microstructure design, traps with cells (cells in green, fluorescent overlay, right); black scale bar 1mm white scale bar 10µm.
• Figure 3: Trap types. Overview of trap microstructure geometries included in our TYC dataset, cropped from high-resolution microscopy images (cf.\ref{['fig:first']}).
• Figure 4: Instance annotations. Brightfield microscopy images including cells & microstructures on the left \ref{['subfig:image']} and the corresponding instance segmentation labels on the right \ref{['subfig:label']}. Shades of grey ( $\blacksquare\!\!\!\!\blacksquare$$\blacksquare\!\!\!\!\blacksquare$$\blacksquare\!\!\!\!\blacksquare$) indicate microstructures (traps) and shades of pink ( $\blacksquare\!\!\!\!\blacksquare$$\blacksquare\!\!\!\!\blacksquare$$\blacksquare\!\!\!\!\blacksquare$) indicate individual cell. For visualization purposes, we adjusted the brightness of the microscopy image and show a crop of the full image. Best viewed in color; zoom in for details.
• Figure 5: Examples of budding yeast cells. A daughter cells (small cells) bud from a trapped mother cells. The bottom row shows the raw image and the top row an overlay between the label and the raw image. Crops from full images are shown.