Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Feasibility study for reconstruction of knee MRI from one corresponding X-ray via CNN

Zhe Wang, Aladine Chetouani, Rachid Jennane

TL;DR

This work addresses early knee OA detection by reframing KL-0 vs KL-2 classification as a multi-level feature learning problem. It introduces Siamese-GAP, a two-branch CNN that fuses low- and high-level features through Global Average Pooling at multiple depths, with ROI-based X-ray patches feeding the model. Ablation shows GAP outperforms GMP, and the method outperforms several baselines and a recent state-of-the-art approach while maintaining a relatively small parameter footprint; Grad-CAM visualizations validate attention to relevant knee regions. The approach promises improved early diagnostic capability on publicly available OA data, with potential generalization to other datasets and broader OA assessment tasks in clinical settings.

Abstract

Generally, X-ray, as an inexpensive and popular medical imaging technique, is widely chosen by medical practitioners. With the development of medical technology, Magnetic Resonance Imaging (MRI), an advanced medical imaging technique, has already become a supplementary diagnostic option for the diagnosis of KOA. We propose in this paper a deep-learning-based approach for generating MRI from one corresponding X-ray. Our method uses the hidden variables of a Convolutional Auto-Encoder (CAE) model, trained for reconstructing X-ray image, as inputs of a generator model to provide 3D MRI.

Feasibility study for reconstruction of knee MRI from one corresponding X-ray via CNN

TL;DR

This work addresses early knee OA detection by reframing KL-0 vs KL-2 classification as a multi-level feature learning problem. It introduces Siamese-GAP, a two-branch CNN that fuses low- and high-level features through Global Average Pooling at multiple depths, with ROI-based X-ray patches feeding the model. Ablation shows GAP outperforms GMP, and the method outperforms several baselines and a recent state-of-the-art approach while maintaining a relatively small parameter footprint; Grad-CAM visualizations validate attention to relevant knee regions. The approach promises improved early diagnostic capability on publicly available OA data, with potential generalization to other datasets and broader OA assessment tasks in clinical settings.

Abstract

Generally, X-ray, as an inexpensive and popular medical imaging technique, is widely chosen by medical practitioners. With the development of medical technology, Magnetic Resonance Imaging (MRI), an advanced medical imaging technique, has already become a supplementary diagnostic option for the diagnosis of KOA. We propose in this paper a deep-learning-based approach for generating MRI from one corresponding X-ray. Our method uses the hidden variables of a Convolutional Auto-Encoder (CAE) model, trained for reconstructing X-ray image, as inputs of a generator model to provide 3D MRI.

Paper Structure

This paper contains 11 sections, 1 equation, 4 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Proposed Approach
  3. Proposed network
  4. Experiments
  5. Public Knee Datasets
  6. Preprocessing
  7. Experimental details
  8. Results and discussion
  9. Ablation study using GAP and GMP
  10. Comparison with state-of-the-art methods
  11. Conclusion and perspectives

Figures (4)

  • Figure 1: Architecture of the proposed Siamese-GAP network. GAP units are represented by red vertical arrows. Concatenation is represented by blue braces. The yellow disk represents the bit-wise addition, and the purple block represents the fully connected and Softmax layers.
  • Figure 2: Classical Siamese Network Ref and add Obtained similarity
  • Figure 3: (a) Original knee radiographs from OAI, extracted ROI in red. (b) Obtained patches in blue and orange boxes. (c) Extracted patches for the two sub-networks respectively
  • Figure 4: Localizations on knee joint images of KL-2 presented by Grad-CAM based on each series of models which archived the best performance. The original image with KL-2 is (a), the attention maps produced from the last CNN layer of corresponding models are (b),(c) and (d), and (e) represents the attention map produced by our model.