Single Tensor Cell Segmentation using Scalar Field Representations

TL;DR

This work reframes cell instance segmentation as learning a single continuous scalar field over an image domain, whose segmentation is obtained via watershed. It introduces two physics-inspired representations—diffusion and Poisson fields—trained with a single-tensor network using an $\ell_1$ loss, yielding sharp, boundary-rich maps. The approach eliminates multi-head prediction and complex post-processing, delivering competitive or superior results across diverse datasets with lower computational and memory demands. The combination of field-based representations and watershed provides a robust, efficient pathway for edge-friendly cell segmentation in challenging microscopy data.

Abstract

We investigate image segmentation of cells under the lens of scalar fields. Our goal is to learn a continuous scalar field on image domains such that its segmentation produces robust instances for cells present in images. This field is a function parameterized by the trained network, and its segmentation is realized by the watershed method. The fields we experiment with are solutions to the Poisson partial differential equation and a diffusion mimicking the steady-state solution of the heat equation. These solutions are obtained by minimizing just the field residuals, no regularization is needed, providing a robust regression capable of diminishing the adverse impacts of outliers in the training data and allowing for sharp cell boundaries. A single tensor is all that is needed to train a \unet\ thus simplifying implementation, lowering training and inference times, hence reducing energy consumption, and requiring a small memory footprint, all attractive features in edge computing. We present competitive results on public datasets from the literature and show that our novel, simple yet geometrically insightful approach can achieve excellent cell segmentation results.

Figures (5)

• Figure 1: Our proposed segmentation scheme is a two--step process. It initially infers field maps from a single tensor model trained on grayscale images and then segment these maps using the watershed method. In the example above, a cluster of cells (A) admits a Poisson field map (B) generated from our inference model, clearly presenting boundaries separating adjacent cells and background. We invert such map while preserving the background mask to obtain an image (C) suitable for watershed segmentation. Regional h-minima are then computed (D) and applied as basins by the watershed method to give the instance segmentation in (E) showing boundaries tightly conforming to cell contours. The background mask from (C) eliminates segmented regions lying on it. The field pseudocolormap in (F) helps visualize the Poisson map in (B) and regional maxima extent (dark regions) leading to the compact basins in (D).
• Figure 2: Ground truth field maps $\{U_k\}$ generated for background and each and all cell instances are used in our $\hbox{$\ell_1$}$ regression. In this illustration, annotation by a biologist yields labeled cell instances shown in the data column. For each cell instance their Poisson, diffusion, and Euclidean distance transform field maps, all normalized in $[0,1]$, are shown together with 3D rendering of their respective topographic maps. Background has a field value of zero. The Poisson and diffusion maps are smooth and they have well localized round peaks, offering overall better images and compact seeds for watershed segmentation.
• Figure 3: Our Poisson model is capable of segmenting well densely packed and touching nuclei. A small crop of a large tonsil nuclei image segmented using a model trained with T--cells data is shown in (A) with its overlaid segmentation in (B). In panel (C) we show the 3D topographic map of the inferred Poisson field for the crop, colored by instance, with the field depicted in the insert as a 2D pseudocolormap. The field in each nucleus instance is sharply defined, note their peaked shapes in (C), with difficulties only when overlapping is extensive, as in the possible example pointed by arrows.
• Figure 4: Test segmentation for SH-SHY5Y label-free cells from LIVECell dataset edlund2021livecell. We enhanced the original phase-contrast image to help visualization of its cells (A) and their segmentations -- manual ground truth (B), diffusion (C), and Poisson (D). All methods have relatively low score (see table \ref{['tab:segmentation_models']}) partially due to questionable ground truth (see arrows), overlaps, and crossovers as illustrated by squares in (A) and (B). Nonetheless, we show here that our method is able to segment cells imaged with challenging phase-contrast signals.
• Figure 5: Example segmentations for datasets in Table.\ref{['tab:segmentation_models']}. Each panel contains, from left to right, small crop of original image, ground truth, Diffusion map and corresponding overlaid colored segmentation, and Poisson map and its segmentation. Datasets are A-172 (A), PhC-C2DH-U373 (B, not on Table.\ref{['tab:segmentation_models']}), Fluo-C2DL-Huh7 (C), DIC-C2DH-HeLa (D), BF-C2DL-HSC (E), and T-Cells (F). The models segment well packed cells and in diverse modalities.