Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Z-upscaling: Optical Flow Guided Frame Interpolation for Isotropic Reconstruction of 3D EM Volumes

Fisseha A. Ferede, Ali Khalighifar, Jaison John, Krishnan Venkataraman, Khaled Khairy

TL;DR

A novel optical flow based approach to enhance the axial resolution of anisotropic 3D EM volumes to achieve isotropic 3D reconstruction and exploits recent state-of-the-art learning methods for video frame interpolation and transfer learning techniques.

Abstract

We propose a novel optical flow based approach to enhance the axial resolution of anisotropic 3D EM volumes to achieve isotropic 3D reconstruction. Assuming spatial continuity of 3D biological structures in well aligned EM volumes, we reasoned that optical flow estimation techniques, often applied for temporal resolution enhancement in videos, can be utilized. Pixel level motion is estimated between neighboring 2D slices along z, using spatial gradient flow estimates to interpolate and generate new 2D slices resulting in isotropic voxels. We leverage recent state-of-the-art learning methods for video frame interpolation and transfer learning techniques, and demonstrate the success of our approach on publicly available ultrastructure EM volumes.

Z-upscaling: Optical Flow Guided Frame Interpolation for Isotropic Reconstruction of 3D EM Volumes

TL;DR

A novel optical flow based approach to enhance the axial resolution of anisotropic 3D EM volumes to achieve isotropic 3D reconstruction and exploits recent state-of-the-art learning methods for video frame interpolation and transfer learning techniques.

Abstract

We propose a novel optical flow based approach to enhance the axial resolution of anisotropic 3D EM volumes to achieve isotropic 3D reconstruction. Assuming spatial continuity of 3D biological structures in well aligned EM volumes, we reasoned that optical flow estimation techniques, often applied for temporal resolution enhancement in videos, can be utilized. Pixel level motion is estimated between neighboring 2D slices along z, using spatial gradient flow estimates to interpolate and generate new 2D slices resulting in isotropic voxels. We leverage recent state-of-the-art learning methods for video frame interpolation and transfer learning techniques, and demonstrate the success of our approach on publicly available ultrastructure EM volumes.

Paper Structure

This paper contains 12 sections, 1 equation, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. Temporal Interpolation
  4. Spatial Interpolation for 3D Isotropic Reconstruction
  5. FILM for 3D Isotropic Reconstruction
  6. Experiments
  7. Dataset
  8. Training Schedule
  9. Evaluation
  10. Discussion
  11. Conclusion
  12. Conflict of Interest

Figures (3)

  • Figure 1: Sample reconstruction of FIB-25 EM data for $\times8$ up sampling. Reconstruction within the red box is zoomed in and overlaid on the green box region for clearer comparison.
  • Figure 2: Architecture of FILM for 3D isotropic reconstruction. Feature extractor,$\tau (\cdot)$, takes multi-resolution pyramid input of a pair of neighbouring $Z$ slices, $(Z_1, Z_3)$ to extract pyramid features. These pyramid features are then used to compute bi-directional spatial flows from unknown frame $Z_2$ to the input slices. Estimated spatial flows are warped onto the input frames and decoded using U-net decoder to reconstruct middle frame output, $Z_2$.
  • Figure 3: Sample visualization results on FIB-25 dataset. The first rows demonstrate the reconstructed z slices along with the corresponding ground truths. The second row shows pixel-wise $L1$ norm error map and PSNR values of the reconstructed frames. White regions in the error map signify erroneous regions.