Tracking one-in-a-million: Large-scale benchmark for microbial single-cell tracking with experiment-aware robustness metrics

TL;DR

The largest publicly available and annotated dataset for MLCI, containing more than 1.4 million cell instances, 29k cell tracks, and 14k cell divisions is presented, which quantifies the influence of experiment parameters on the tracking quality, and gives the opportunity to develop new data-driven methods that generalize across imaging and experiment parameters.

Abstract

Tracking the development of living cells in live-cell time-lapses reveals crucial insights into single-cell behavior and presents tremendous potential for biomedical and biotechnological applications. In microbial live-cell imaging (MLCI), a few to thousands of cells have to be detected and tracked within dozens of growing cell colonies. The challenge of tracking cells is heavily influenced by the experiment parameters, namely the imaging interval and maximal cell number. For now, tracking benchmarks are not widely available in MLCI and the effect of these parameters on the tracking performance are not yet known. Therefore, we present the largest publicly available and annotated dataset for MLCI, containing more than 1.4 million cell instances, 29k cell tracks, and 14k cell divisions. With this dataset at hand, we generalize existing tracking metrics to incorporate relevant imaging and experiment parameters into experiment-aware metrics. These metrics reveal that current cell tracking methods crucially depend on the choice of the experiment parameters, where their performance deteriorates at high imaging intervals and large cell colonies. Thus, our new benchmark quantifies the influence of experiment parameters on the tracking quality, and gives the opportunity to develop new data-driven methods that generalize across imaging and experiment parameters. The benchmark dataset is publicly available at https://zenodo.org/doi/10.5281/zenodo.7260136.

Figures (4)

• Figure 1: Data acquisition in microbial live-cell imaging. A microfluidic cultivation device is mounted to an automated microscope (A, B). The device contains hundreds to thousands of rectangular cultivation chambers (C) that are imaged one-by-one, moving the microscope stage and capturing images at a series of time-points (D). This imaging-and-movement loop, indicated by the purple line (C), is repeated with a pre-set imaging interval and leads to time-lapse recordings capturing the temporal development of the cell colony from a few cells up to several thousands (E). The images in (E) depict a $80 \times 90 \mu m$ region. The figure is adapted from Blöbaum et al.blobaum_quantifying_2024.
• Figure 2: Temporal imbalance of the five microbial time-lapse recordings of the benchmark dataset. We measured the temporal development of cell count (A), cell divisions (B), and cell disappearance events (C) for each time-lapse. Cell division and disappearance events are grouped into bins of 100 minutes. The dashed lines in (A) indicate cell count limits (100, 400, 700, 1,000, 1,300, 1,600).
• Figure 3: Subsampling and cell count limiting of time-lapse sequences. (A) shows an excerpt of an MLCI time-lapse. (B) shows an exemplary subsampling with a factor of $3$ and truncation at a cell count limit of $21$ leading to $3$ frames in total. Grayed out images denote frames removed due to subsampling, the dashed box denotes the cell count limit. (C-E) shows the temporal changes in the number of cell division, disappearance and movement per microscopy frame when subsampling to different imaging intervals. The curves have been exponentially smoothed. (F) shows the percentage of division events in contrast to non-division links. (C-F) show data from the TOIAM test split.
• Figure 4: The EATM based on the DIV metric measured across different imaging intervals and cell count limits. The black line marks the region surpassing the $80~\%$ threshold ($RM@0.8$). The higher the value of the DIV metric, the better is the reconstruction of the cell divisions, with a value of $1$ meaning perfect cell division reconstruction. Evaluations have been performed on the test split.