Deep Temporal Sequence Classification and Mathematical Modeling for Cell Tracking in Dense 3D Microscopy Videos of Bacterial Biofilms

TL;DR

DenseTrack addresses automatic cell tracking in dense 3D time-lapse microscopy of bacterial biofilms. It integrates a deep temporal sequence classifier for frame-by-frame association with a constrained one-to-one matching optimization, and introduces an eigendecomposition-based strategy for detecting cell divisions. Training relies on synthetically generated biofilm sequences due to limited ground truth, and evaluation covers both synthetic and real lattice-light-sheet videos of E. coli and S.1oneidensis, revealing superior TRA and Division-F1 compared to state-of-the-art methods. Ablation studies validate the benefits of incorporating near-temporal history and the InceptionTime-based classifier, demonstrating robustness across different densities and division rates.

Abstract

Automatic cell tracking in dense environments is plagued by inaccurate correspondences and misidentification of parent-offspring relationships. In this paper, we introduce a novel cell tracking algorithm named DenseTrack, which integrates deep learning with mathematical model-based strategies to effectively establish correspondences between consecutive frames and detect cell division events in crowded scenarios. We formulate the cell tracking problem as a deep learning-based temporal sequence classification task followed by solving a constrained one-to-one matching optimization problem exploiting the classifier's confidence scores. Additionally, we present an eigendecomposition-based cell division detection strategy that leverages knowledge of cellular geometry. The performance of the proposed approach has been evaluated by tracking densely packed cells in 3D time-lapse image sequences of bacterial biofilm development. The experimental results on simulated as well as experimental fluorescence image sequences suggest that the proposed tracking method achieves superior performance in terms of both qualitative and quantitative evaluation measures compared to recent state-of-the-art cell tracking approaches.

Figures (9)

• Figure 1: Overview of the proposed tracking approach DenseTrack. In (a) and (b), we depict our frame-by-frame matching technique, which entails calculating deep learning-based association scores and integrating them into one-to-one matching optimization. (c) illustrates the detection of a cell division event by identifying the neighboring instance with the minimum projection along the $2^{nd}$ and $3^{rd}$ principal components of the unmatched instance in frame $t+1$.
• Figure 2: Qualitative visualization of cell tracking by DenseTrack in a synthetic biofilm sequence with 50 frames captured at 10 seconds frame interval. We demonstrate tracking of three particular cells at several frames in the sequence. Each 3D frame is displayed as a maximum intensity projection along z axis.
• Figure 3: Showing evidence of effective cell division detection over time by the DenseTrack method through (a) space-time plot and (b) volume-over-time plot, demonstrated for the 'blue' cell in the synthetic sequence in Fig. \ref{['fig_syn_track_qual']}.
• Figure 4: Qualitative observation of bacterial cell tracking using the DenseTrack method in a real S. oneidensis biofilm sequence, consisting of 30 frames captured at 30-second intervals. We display the predicted matched instances for a group of randomly selected cells over various time points in the video, each represented by a distinct color.
• Figure 5: Visualizing thirty predicted trajectories of the S. oneidensis sequence obtained from (a) the $\textit{DenseTrack}$ method and (b) the $\textit{Ultrack}$ method, in comparison to the corresponding manually labeled ground truth trajectories. The spatial dimension is $244\times 262\times 87$ voxels in $x$-$y$-$z$. The trajectories depicted in Fig.\ref{['shewn_space_trajectory_plot_proposed']} exhibit greater alignment with the ground truth.