Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Agent with Warm Start and Active Termination for Plane Localization in 3D Ultrasound

Haoran Dou, Xin Yang, Jikuan Qian, Wufeng Xue, Hao Qin, Xu Wang, Lequan Yu, Shujun Wang, Yi Xiong, Pheng-Ann Heng, Dong Ni

TL;DR

A novel reinforcement learning (RL) framework to automatically localize fetal brain standard planes in 3D US using a recurrent neural network based strategy for active termination of the agent's interaction procedure, which improves both the accuracy and efficiency of the localization system.

Abstract

Standard plane localization is crucial for ultrasound (US) diagnosis. In prenatal US, dozens of standard planes are manually acquired with a 2D probe. It is time-consuming and operator-dependent. In comparison, 3D US containing multiple standard planes in one shot has the inherent advantages of less user-dependency and more efficiency. However, manual plane localization in US volume is challenging due to the huge search space and large fetal posture variation. In this study, we propose a novel reinforcement learning (RL) framework to automatically localize fetal brain standard planes in 3D US. Our contribution is two-fold. First, we equip the RL framework with a landmark-aware alignment module to provide warm start and strong spatial bounds for the agent actions, thus ensuring its effectiveness. Second, instead of passively and empirically terminating the agent inference, we propose a recurrent neural network based strategy for active termination of the agent's interaction procedure. This improves both the accuracy and efficiency of the localization system. Extensively validated on our in-house large dataset, our approach achieves the accuracy of 3.4mm/9.6° and 2.7mm/9.1° for the transcerebellar and transthalamic plane localization, respectively. Ourproposed RL framework is general and has the potential to improve the efficiency and standardization of US scanning.

Agent with Warm Start and Active Termination for Plane Localization in 3D Ultrasound

TL;DR

A novel reinforcement learning (RL) framework to automatically localize fetal brain standard planes in 3D US using a recurrent neural network based strategy for active termination of the agent's interaction procedure, which improves both the accuracy and efficiency of the localization system.

Abstract

Standard plane localization is crucial for ultrasound (US) diagnosis. In prenatal US, dozens of standard planes are manually acquired with a 2D probe. It is time-consuming and operator-dependent. In comparison, 3D US containing multiple standard planes in one shot has the inherent advantages of less user-dependency and more efficiency. However, manual plane localization in US volume is challenging due to the huge search space and large fetal posture variation. In this study, we propose a novel reinforcement learning (RL) framework to automatically localize fetal brain standard planes in 3D US. Our contribution is two-fold. First, we equip the RL framework with a landmark-aware alignment module to provide warm start and strong spatial bounds for the agent actions, thus ensuring its effectiveness. Second, instead of passively and empirically terminating the agent inference, we propose a recurrent neural network based strategy for active termination of the agent's interaction procedure. This improves both the accuracy and efficiency of the localization system. Extensively validated on our in-house large dataset, our approach achieves the accuracy of 3.4mm/9.6° and 2.7mm/9.1° for the transcerebellar and transthalamic plane localization, respectively. Ourproposed RL framework is general and has the potential to improve the efficiency and standardization of US scanning.

Paper Structure

This paper contains 10 sections, 2 equations, 5 figures, 1 table.

Table of Contents

  1. Introduction
  2. Methodology
  3. Deep Reinforcement Learning Framework for Plane Localization
  4. Landmark-aware Plane Alignment for Warm Start
  5. Recurrent Neural Network based Active Termination
  6. Experimental Results
  7. Materials and Implementation Details.
  8. Quantitative and Qualitative Analysis:
  9. Conclusion
  10. Acknowledgments:

Figures (5)

  • Figure 1: Fetal brain planes in 3D US. (a) Blue lines show transthalamic (TT) and transcerebellar (TC) plane positions. Red dots (left to right) show three landmarks: genu of corpus callosum, splenium of corpus callosum, cerebellar vermis. (b)(c) Example planes from two volumes illustrate the huge search space and large fetal posture variation.
  • Figure 2: Schematic view of our proposed framework.
  • Figure 3: Landmarks of 100 US volumes (left) aligned to a place-specific atlas space (middle) provides strong spatial bounds for RL agent actions (right). Red, green and blue dots indicate landmarks shown in Fig. 1(a).
  • Figure 4: Mean Q-value of 8 action candidates (yellow) and ADI (blue) on training dataset. Green point denotes the optimal termination step with maximum ADI.
  • Figure 5: TC (left) and TT (right) results. Top row: ground truth (left) and predicted (right) plane. Bottom row: left, active termination step (dotted red line) compared to optimal step in green dot, 3D visualization of ground truth and predicted plane (right).