Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Transferable Optimization Network for Cross-Domain Image Reconstruction

Yunmei Chen, Chi Ding, Xiaojing Ye

TL;DR

A novel transfer learning framework to tackle the challenge of limited training data in image reconstruction problems by training a powerful universal feature-extractor and a task-specific domain-adapter for a new target domain or task with only a limited amount of data available for training.

Abstract

We develop a novel transfer learning framework to tackle the challenge of limited training data in image reconstruction problems. The proposed framework consists of two training steps, both of which are formed as bi-level optimizations. In the first step, we train a powerful universal feature-extractor that is capable of learning important knowledge from large, heterogeneous data sets in various domains. In the second step, we train a task-specific domain-adapter for a new target domain or task with only a limited amount of data available for training. Then the composition of the adapter and the universal feature-extractor effectively explores feature which serve as an important component of image regularization for the new domains, and this leads to high-quality reconstruction despite the data limitation issue. We apply this framework to reconstruct under-sampled MR images with limited data by using a collection of diverse data samples from different domains, such as images of other anatomies, measurements of various sampling ratios, and even different image modalities, including natural images. Experimental results demonstrate a promising transfer learning capability of the proposed method.

Transferable Optimization Network for Cross-Domain Image Reconstruction

TL;DR

A novel transfer learning framework to tackle the challenge of limited training data in image reconstruction problems by training a powerful universal feature-extractor and a task-specific domain-adapter for a new target domain or task with only a limited amount of data available for training.

Abstract

We develop a novel transfer learning framework to tackle the challenge of limited training data in image reconstruction problems. The proposed framework consists of two training steps, both of which are formed as bi-level optimizations. In the first step, we train a powerful universal feature-extractor that is capable of learning important knowledge from large, heterogeneous data sets in various domains. In the second step, we train a task-specific domain-adapter for a new target domain or task with only a limited amount of data available for training. Then the composition of the adapter and the universal feature-extractor effectively explores feature which serve as an important component of image regularization for the new domains, and this leads to high-quality reconstruction despite the data limitation issue. We apply this framework to reconstruct under-sampled MR images with limited data by using a collection of diverse data samples from different domains, such as images of other anatomies, measurements of various sampling ratios, and even different image modalities, including natural images. Experimental results demonstrate a promising transfer learning capability of the proposed method.
Paper Structure (31 sections, 3 theorems, 51 equations, 7 figures, 6 tables, 1 algorithm)

This paper contains 31 sections, 3 theorems, 51 equations, 7 figures, 6 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Paper organization
  3. Notations
  4. Related work
  5. Our contributions
  6. Method
  7. Step 1: Train the feature-extractor $\mathbf{g}$
  8. Step 2: Train the adapters $\{\hat{\mathbf{h}}_j: j \in [J]\}$
  9. Testing
  10. Algorithm
  11. Modified efficient learnable descent algorithm
  12. Convergence analysis
  13. Unrolling network of modified ELDA for TL
  14. Experimental results
  15. Implementation details
  16. ...and 16 more sections

Key Result

Lemma 4.4

The gradient $\nabla r_{\varepsilon}$ is $(\sqrt{d_1} L_\mathbf{q}+\frac{M^2}{\varepsilon})$-Lipschitz continuous. Hence, $\nabla \phi_\varepsilon(\mathbf{x})$ is $L_{\varepsilon}$-Lipschitz continuous with $L_{\varepsilon}:= L_f + \sqrt{d_1}L_\mathbf{q} + \frac{M^2}{\varepsilon} = O(\varepsilon^{-1

Figures (7)

  • Figure 1: The flowchart (top) of the $t$th iteration of Algorithm \ref{['alg:elda']}, the $\mathbf{u}$-module (middle), and the $\mathbf{v}$-module (bottom). Each round node represents a variable. Each rectangular node represents a module, mapping, or condition check. The identity mapping is denoted by Id.
  • Figure 2: Comparison of cross-anatomy TL reconstruction results using 20% Cartesian sampling ratio in Section \ref{['subsubsec:cross-anatomy']}. Top two rows show two instances of reconstructed cardiac images. Bottom rows show two instances of reconstructed prostate images. Blue squares zoom in the corresponding small squares in the images.
  • Figure 3: Comparison of cross-sampling-ratio TL reconstruction results in Section \ref{['subsubsec:cross-sampling']}. Top two rows show two instances of reconstructed images under 15% sampling ratio. Bottom rows show two instances of reconstructed images under 25% sampling ratio. Blue squares zoom in the corresponding small squares in the images.
  • Figure 4: Comparison of cross-modality reconstruction results in Section \ref{['subsubsec:cross-modality']}. Knowledge is transferred from natural image datasets ImageNet and CIFAR-10 to medical image reconstruction. Top two rows show two instances of reconstructed images in the fastMRI dataset. Bottom rows show two instances of reconstructed images in the Stanford2D dataset. Blue squares zoom in the corresponding small squares in the images.
  • Figure 5: Reconstructed images, zoom-in views, and pointwise absolute errors obtained by the compared methods on the cross-anatomy experiment (top three rows), cross-sampling-ratio experiment (middle three rows), and cross-modality experiment (bottom three rows).
  • ...and 2 more figures

Theorems & Definitions (7)

  • Definition 4.1: Clarke subdifferential
  • Definition 4.2: Clarke stationary point
  • Lemma 4.4: Lemma 3.2 chen2021learnable
  • Lemma 4.5
  • proof
  • Theorem 4.6
  • proof