Unsupervised Learning for Feature Extraction and Temporal Alignment of 3D+t Point Clouds of Zebrafish Embryos

TL;DR

Problem: align developmental stages across zebrafish embryos from 3D+t point clouds without manual labels. Approach: a FoldingNet-based autoencoder learns latent descriptors from each frame, and a regression network maps these features to developmental time, with a postprocessing step to ensure monotonic alignments. Contributions: (i) spherical initial template and Modified Chamfer Distance for improved reconstruction, (ii) a regression-based temporal alignment framework validated with synthetic ground-truth, and (iii) demonstrated robustness to rotations and a fully-unsupervised workflow. Findings: achieves an average mismatch of $3.83$ minutes over $5.3$ hours and scales without labeling, enabling bias-free comparisons and potential atlas construction in zebrafish development.

Abstract

Zebrafish are widely used in biomedical research and developmental stages of their embryos often need to be synchronized for further analysis. We present an unsupervised approach to extract descriptive features from 3D+t point clouds of zebrafish embryos and subsequently use those features to temporally align corresponding developmental stages. An autoencoder architecture is proposed to learn a descriptive representation of the point clouds and we designed a deep regression network for their temporal alignment. We achieve a high alignment accuracy with an average mismatch of only 3.83 minutes over an experimental duration of 5.3 hours. As a fully-unsupervised approach, there is no manual labeling effort required and unlike manual analyses the method easily scales. Besides, the alignment without human annotation of the data also avoids any influence caused by subjective bias.

Figures (10)

• Figure 1: Comparison of raw 3D point clouds (left sub-panels) and the reconstructions of our modified FoldingNet that was trained with MCD and the spherical point template (right sub-panels). Shape and density distribution of the reconstructions are nicely preserved, i.e., the learned representation successfully condenses the properties of the input point clouds (see Suppl. \ref{['fig:draw_rec']} for additional examples).
• Figure 1: Average values of the coordinates along x- and y-axis of the input point cloud (blue) and the corresponding reconstruction of a FoldingNet (orange) that was trained either with CD and MCD. Upon changes of the center of gravity over time, the output point cloud of MCD is more consistent with the input. This indicates that a FoldingNet trained with MCD outperforms CD in the reconstruction quality.
• Figure 2: The temporal alignment network is trained with a baseline embryo and learns to map each feature vector to the corresponding reference frame index (left). The trained network is then applied to feature vectors of a new embryo and the predicted frame index sequence indicates how the embryo should be aligned to the reference (right).
• Figure 2: Continuation of \ref{['fig:draw_rec_main']} in the main text.
• Figure 3: Alignment results of the embryos with different shifting methods. The black line indicates the average result of all experiments and the area shaded in gray represents the variance. The alignment error is calculated as the average number of mismatched time frame indices.