Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MPBD-LSTM: A Predictive Model for Colorectal Liver Metastases Using Time Series Multi-phase Contrast-Enhanced CT Scans

Xueyang Li, Han Xiao, Weixiang Weng, Xiaowei Xu, Yiyu Shi

TL;DR

This work addresses early CRLM detection from time-series 5D CECT data, introducing MPBD-LSTM, a phase-aware, multi-plane extension of E3D-LSTM that processes arterial and portal venous phases in parallel while employing bi-directional temporal modeling. The authors curate a 5D CECT CRLM dataset from two hospitals, build a dedicated MPBD-LSTM architecture with two planes and two bi-directional 3D-LSTM stacks per plane, and demonstrate superior predictive performance (AUC 0.790) over several baselines. Ablation studies confirm the benefits of bi-directional connections and parallel multi-phase processing, while timestamp and phase analyses reveal that combining all timestamps across A and V phases yields the strongest signal, with the V phase contributing more than the A phase. The results highlight the potential of phase-aware spatiotemporal modeling for CRLM prediction, though the authors acknowledge remaining challenges from inter-patient variability and 5D feature fusion that warrant further research.

Abstract

Colorectal cancer is a prevalent form of cancer, and many patients develop colorectal cancer liver metastasis (CRLM) as a result. Early detection of CRLM is critical for improving survival rates. Radiologists usually rely on a series of multi-phase contrast-enhanced computed tomography (CECT) scans done during follow-up visits to perform early detection of the potential CRLM. These scans form unique five-dimensional data (time, phase, and axial, sagittal, and coronal planes in 3D CT). Most of the existing deep learning models can readily handle four-dimensional data (e.g., time-series 3D CT images) and it is not clear how well they can be extended to handle the additional dimension of phase. In this paper, we build a dataset of time-series CECT scans to aid in the early diagnosis of CRLM, and build upon state-of-the-art deep learning techniques to evaluate how to best predict CRLM. Our experimental results show that a multi-plane architecture based on 3D bi-directional LSTM, which we call MPBD-LSTM, works best, achieving an area under curve (AUC) of 0.79. On the other hand, analysis of the results shows that there is still great room for further improvement.

MPBD-LSTM: A Predictive Model for Colorectal Liver Metastases Using Time Series Multi-phase Contrast-Enhanced CT Scans

TL;DR

This work addresses early CRLM detection from time-series 5D CECT data, introducing MPBD-LSTM, a phase-aware, multi-plane extension of E3D-LSTM that processes arterial and portal venous phases in parallel while employing bi-directional temporal modeling. The authors curate a 5D CECT CRLM dataset from two hospitals, build a dedicated MPBD-LSTM architecture with two planes and two bi-directional 3D-LSTM stacks per plane, and demonstrate superior predictive performance (AUC 0.790) over several baselines. Ablation studies confirm the benefits of bi-directional connections and parallel multi-phase processing, while timestamp and phase analyses reveal that combining all timestamps across A and V phases yields the strongest signal, with the V phase contributing more than the A phase. The results highlight the potential of phase-aware spatiotemporal modeling for CRLM prediction, though the authors acknowledge remaining challenges from inter-patient variability and 5D feature fusion that warrant further research.

Abstract

Colorectal cancer is a prevalent form of cancer, and many patients develop colorectal cancer liver metastasis (CRLM) as a result. Early detection of CRLM is critical for improving survival rates. Radiologists usually rely on a series of multi-phase contrast-enhanced computed tomography (CECT) scans done during follow-up visits to perform early detection of the potential CRLM. These scans form unique five-dimensional data (time, phase, and axial, sagittal, and coronal planes in 3D CT). Most of the existing deep learning models can readily handle four-dimensional data (e.g., time-series 3D CT images) and it is not clear how well they can be extended to handle the additional dimension of phase. In this paper, we build a dataset of time-series CECT scans to aid in the early diagnosis of CRLM, and build upon state-of-the-art deep learning techniques to evaluate how to best predict CRLM. Our experimental results show that a multi-plane architecture based on 3D bi-directional LSTM, which we call MPBD-LSTM, works best, achieving an area under curve (AUC) of 0.79. On the other hand, analysis of the results shows that there is still great room for further improvement.

Paper Structure

This paper contains 13 sections, 2 equations, 2 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Dataset and Methodology
  3. Dataset
  4. Methods
  5. MPBD-LSTM
  6. Experiments
  7. Data augmentation and selection
  8. Experiment setup
  9. Results and Discussion
  10. Ablation study on model structures
  11. Ablation study on timestamps and phases
  12. Error Analysis
  13. Conclusion

Figures (2)

  • Figure 1: Representative slices from 3D CT images of different patients in our dataset, at A/V phases and timestamps T0, T1, T2 (cropped to 256x256 for better view).
  • Figure 2: (a) The general structure of MPBD-LSTM. The yellow plane is the 1st plane which is used to process the portal venous phase CT scans, and the gray plane is the second one used to process the arterial phase CT scans. $\mu$ is the average function. (b) The inner structure of a 3D-LSTM module. Blue arrow stands for the forward pass which generates the output of $\overrightarrow{h}_{v,t_0}$ and red arrow indicates the backward pass generating the output of $\overleftarrow{h}_{v,t_0}$. $\sigma$ is the function used to combine two hidden-state outputs. $y_{v,t_0}$ is the output of this 3D-LSTM module after processed by $\sigma$.