Echocardiography to Cardiac MRI View Transformation for Real-Time Blind Restoration

TL;DR

This paper tackles the challenge of corrupted echocardiography by proposing a blind restoration approach that learns a one-to-one transformation from 4-chamber echocardiography to cardiac MRI views using Cycle-GANs. It introduces the Echo2MRI dataset and integrates temporal information via 3-channel frame augmentation to reduce flicker, enabling synthetic MRI views with improved contrast and boundary definition. Domain experts demonstrate that the synthetic MRI views are largely indistinguishable from real MRIs and can support RWMA-based diagnosis in a majority of cases, with real-time inference achievable on GPUs. The work has potential to democratize access to MRI-like cardiac imaging, enabling more accurate and timely diagnoses while reducing cost, though further work is needed to perfect temporal coherence and edge-case reliability.

Abstract

Echocardiography is the most widely used imaging to monitor cardiac functions, serving as the first line in early detection of myocardial ischemia and infarction. However, echocardiography often suffers from several artifacts including sensor noise, lack of contrast, severe saturation, and missing myocardial segments which severely limit its usage in clinical diagnosis. In recent years, several machine learning methods have been proposed to improve echocardiography views. Yet, these methods usually address only a specific problem (e.g. denoising) and thus cannot provide a robust and reliable restoration in general. On the other hand, cardiac MRI provides a clean view of the heart without suffering such severe issues. However, due to its significantly higher cost, it is often only afforded by a few major hospitals, hence hindering its use and accessibility. In this pilot study, we propose a novel approach to transform echocardiography into the cardiac MRI view. For this purpose, Echo2MRI dataset, consisting of echocardiography and real cardiac MRI image pairs, is composed and will be shared publicly. A dedicated Cycle-consistent Generative Adversarial Network (Cycle-GAN) is trained to learn the transformation from echocardiography frames to cardiac MRI views. An extensive set of qualitative evaluations shows that the proposed transformer can synthesize high-quality artifact-free synthetic cardiac MRI views from a given sequence of echocardiography frames. Medical evaluations performed by a group of cardiologists further demonstrate that synthetic MRI views are indistinguishable from their original counterparts and are preferred over their initial sequence of echocardiography frames for diagnosis in 78.9% of the cases.

Figures (45)

• Figure 1: Sample echocardiography frames from the Echo2MRI dataset demonstrating poor quality due to common artifacts such as saturation (SAT), out-of-view (OOV), noise, and lack of contrast (LOC).
• Figure 2: Comparison of 4-chamber Echocardiography (A), generated synthetic cardiac MRI view (B), and the original cardiac MRI view (C) across different phases of the cardiac cycles. The generated synthetic cardiac MRI views successfully interpolate the LV wall's missing segments while preserving the cardiac motion's temporal coherence.
• Figure 3: Illustration of pre- and post-processing steps. Preprocessing involes encapsulating temporal information, $t-1$, $t$ and, $t+1$, into a single 3-channel image. After translating from the echocardiography ($X$) domain to the MRI ($Y$) domain with the Generator $G$, the resulting frames also retain temporal information. In the post-processing, output frame is obtained by averaging the slices of timestamps from consecutive frames.
• Figure 4: Cycle-GAN architecture for image-to-image translation between Source Domain $X$, and Target Domain $Y$, corresponding to echocardiography to cardiac MRI. The generator $G:X\rightarrow Y$ translates echocardiography images to the cardiac MRI domain, whereas the generator $F:Y\rightarrow X$ performs the inverse translation from cardiac MRI to echocardiography. Discriminators $D_X$ and $D_Y$ distinguish real and generated images in the respective domains. Losses $\mathcal{L}_{A}$ and $\mathcal{L}_{\text{cyc}}$ are defined in \ref{['eq:gen-cost1']} and \ref{['eq:cyc-cost']} respectively.
• Figure 5: Architecture of the Cycle-GAN generator. The input is a 3-channel image of size $256\times256$, processed through convolutional layers. Between each layer, indicated by arrows, instance normalization and ReLU activations are applied. The input image is processed through the downsampling stage, followed by a sequence of residual (ResNet) blocks repeated 9 times. Then, the image is upsampled back to its original size, completing the transformation.