SparseSSP: 3D Subcellular Structure Prediction from Sparse-View Transmitted Light Images

TL;DR

This work tackles the inefficiency and phototoxicity of traditional fluorescence-based SSP by predicting 3D subcellular fluorescence from sparse TL images. It introduces SparseSSP, a hybrid 3D-2D framework that performs one-to-many Z-axis mapping and folds Z information into channel features, enabling efficient 2D decoding for 3D SSP. The paper also presents prefix and postfix interpolation strategies and dimension transformation mechanisms (depth-to-channel, channel-to-depth) to support various topology configurations, and demonstrates strong performance with reduced imaging frequency on the AllenCell dataset. The approach achieves state-of-the-art or near-state-of-the-art results while significantly lowering computational and imaging demands, with practical implications for monitoring rapid biological dynamics on accessible hardware.

Abstract

Traditional fluorescence staining is phototoxic to live cells, slow, and expensive; thus, the subcellular structure prediction (SSP) from transmitted light (TL) images is emerging as a label-free, faster, low-cost alternative. However, existing approaches utilize 3D networks for one-to-one voxel level dense prediction, which necessitates a frequent and time-consuming Z-axis imaging process. Moreover, 3D convolutions inevitably lead to significant computation and GPU memory overhead. Therefore, we propose an efficient framework, SparseSSP, predicting fluorescent intensities within the target voxel grid in an efficient paradigm instead of relying entirely on 3D topologies. In particular, SparseSSP makes two pivotal improvements to prior works. First, SparseSSP introduces a one-to-many voxel mapping paradigm, which permits the sparse TL slices to reconstruct the subcellular structure. Secondly, we propose a hybrid dimensions topology, which folds the Z-axis information into channel features, enabling the 2D network layers to tackle SSP under low computational cost. We conduct extensive experiments to validate the effectiveness and advantages of SparseSSP on diverse sparse imaging ratios, and our approach achieves a leading performance compared to pure 3D topologies. SparseSSP reduces imaging frequencies compared to previous dense-view SSP (i.e., the number of imaging is reduced up to 87.5% at most), which is significant in visualizing rapid biological dynamics on low-cost devices and samples.

Figures (6)

• Figure 1: Comparisons on SSP implementations. (a) Multi-Net adopts a set of independent 3D models for SSP. (b) Task embedding is an improved strategy which can learn all tasks relied on the one-hot encoding, however, this is still require a dense imaging procedure. Prolonged imaging time and high computational cost are unfriendly. (c) SparseSSP implements an efficient hybrid dimensions topology which only requires less TL imaging slices to achieve the excellent performance.
• Figure 2: Optional strategies in SparseSSP. (a) demonstrates two strategies for one-to-many mapping in the Z-axis. (b) shows four combinations for different dimension encoders and decoders. (c) demonstrates diverse approaches to implementing the dimension transformation.
• Figure 3: Overview of SparseSSP with 3-to-2D topology and prefix interpolation. It is an example following the DoDNet DoDNet style that utilizes the task controller and dynamic head. The framework's task embedding approach can be conveniently changed to other technologies.
• Figure 4: Trend of $R^2$ value as sparsity ratio increased from 2 to 8. Hybrid dimensions topology 3-to-2D (i.e., the blue lines in the figure) shows a slower decay and higher global scores than pure 3D topology (i.e., the orange lines).
• Figure 5: Visualization of subcellular structure prediction. A stereoscopic volume display from a top-down Z-axis view. We compare results of four structures using dense-view and sparse-view TL images at different sparsity ratios. The second column shows results from RepMode (best for dense-view SSP), and columns three to six show results from Tgnet with the best 3-to-2D topology for sparse-view.