D-SarcNet: A Dual-stream Deep Learning Framework for Automatic Analysis of Sarcomere Structures in Fluorescently Labeled hiPSC-CMs

TL;DR

D-SarcNet is proposed, a dual-stream deep learning framework that takes fluorescent hiPSC-CM single-cell images as input and outputs the stage of the sarcomere structural organization on a scale from 1.0 to 5.0, establishing a new state-of-the-art in sarcomere structure assessment from hiPSC-CM images.

Abstract

Human-induced pluripotent stem cell-derived cardiomyocytes (hiPSC-CMs) are a powerful tool in advancing cardiovascular research and clinical applications. The maturation of sarcomere organization in hiPSC-CMs is crucial, as it supports the contractile function and structural integrity of these cells. Traditional methods for assessing this maturation like manual annotation and feature extraction are labor-intensive, time-consuming, and unsuitable for high-throughput analysis. To address this, we propose D-SarcNet, a dual-stream deep learning framework that takes fluorescent hiPSC-CM single-cell images as input and outputs the stage of the sarcomere structural organization on a scale from 1.0 to 5.0. The framework also integrates Fast Fourier Transform (FFT), deep learning-generated local patterns, and gradient magnitude to capture detailed structural information at both global and local levels. Experiments on a publicly available dataset from the Allen Institute for Cell Science show that the proposed approach not only achieves a Spearman correlation of 0.868 marking a 3.7% improvement over the previous state-of-the-art but also significantly enhances other key performance metrics, including MSE, MAE, and R2 score. Beyond establishing a new state-of-the-art in sarcomere structure assessment from hiPSC-CM images, our ablation studies highlight the significance of integrating global and local information to enhance deep learning networks ability to discern and learn vital visual features of sarcomere structure.

Figures (5)

• Figure 1: The D-SarcNet framework for scoring sarcomere structural organization in fluorescently labeled hiPSC-CM single-cell images consists of two primary streams. The first stream, ConvNeXt, processes raw images to extract global features directly from the original images. Simultaneously, the second stream, Swin Transformer, analyzes the corresponding three-channel image — created by stacking representations generated by FFT, the local pattern model, and the Sobel operator — to capture local features. A blocks-combined architecture then integrates feature maps from both streams across various scales to output a score of $\alpha$-actinin-2 pattern structure on a scale from 1.0 to 5.0.
• Figure 2: Illustration of the ConvNeXt framework. The whole framework has been applied to train the local-pattern model, and the ConvNeXt block has been utilized in the D-SarcNet model.
• Figure 3: The Swin Transformer Block in the D-SarcNet model is made up of a linear embedding layer (for the first block) or a patch merging layer (for the other blocks), followed by two Transformer blocks.
• Figure 4: Representative examples of hiPSC-CMs single-cell images and local patterns for each category
• Figure 5: Quantitative measurement of sarcomere properties, including sarcomere length, sarcomere width, and OOP, on the two groups: the first group with predicted scores from 3.5 to 4.0 and the second group with predicted scores from 4.0 to 5.0. (A) Histograms on the first group (blue) and the second group (orange). (B) Box plots and the p-values are represented above the box plots on the two groups.