Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Autonomous Navigation of an Ultrasound Probe Towards Standard Scan Planes with Deep Reinforcement Learning

Keyu Li, Jian Wang, Yangxin Xu, Hao Qin, Dongsheng Liu, Li Liu, Max Q. -H. Meng

TL;DR

A deep reinforcement learning framework to autonomously control the 6-D pose of a virtual US probe based on real-time image feedback to navigate towards the standard scan planes under the restrictions in real-world US scans is proposed.

Abstract

Autonomous ultrasound (US) acquisition is an important yet challenging task, as it involves interpretation of the highly complex and variable images and their spatial relationships. In this work, we propose a deep reinforcement learning framework to autonomously control the 6-D pose of a virtual US probe based on real-time image feedback to navigate towards the standard scan planes under the restrictions in real-world US scans. Furthermore, we propose a confidence-based approach to encode the optimization of image quality in the learning process. We validate our method in a simulation environment built with real-world data collected in the US imaging of the spine. Experimental results demonstrate that our method can perform reproducible US probe navigation towards the standard scan plane with an accuracy of $4.91mm/4.65^\circ$ in the intra-patient setting, and accomplish the task in the intra- and inter-patient settings with a success rate of $92\%$ and $46\%$, respectively. The results also show that the introduction of image quality optimization in our method can effectively improve the navigation performance.

Autonomous Navigation of an Ultrasound Probe Towards Standard Scan Planes with Deep Reinforcement Learning

TL;DR

A deep reinforcement learning framework to autonomously control the 6-D pose of a virtual US probe based on real-time image feedback to navigate towards the standard scan planes under the restrictions in real-world US scans is proposed.

Abstract

Autonomous ultrasound (US) acquisition is an important yet challenging task, as it involves interpretation of the highly complex and variable images and their spatial relationships. In this work, we propose a deep reinforcement learning framework to autonomously control the 6-D pose of a virtual US probe based on real-time image feedback to navigate towards the standard scan planes under the restrictions in real-world US scans. Furthermore, we propose a confidence-based approach to encode the optimization of image quality in the learning process. We validate our method in a simulation environment built with real-world data collected in the US imaging of the spine. Experimental results demonstrate that our method can perform reproducible US probe navigation towards the standard scan plane with an accuracy of in the intra-patient setting, and accomplish the task in the intra- and inter-patient settings with a success rate of and , respectively. The results also show that the introduction of image quality optimization in our method can effectively improve the navigation performance.

Paper Structure

This paper contains 19 sections, 7 equations, 4 figures, 1 table, 1 algorithm.

Table of Contents

  1. INTRODUCTION
  2. RELATED WORK
  3. METHODS
  4. Reinforcement Learning for Probe Navigation
  5. States
  6. Actions
  7. State transition under restrictions
  8. Reward function
  9. Termination conditions
  10. Confidence-aware Agent
  11. Deep Q-Network Training
  12. Deep Q-Learning Algorithm
  13. Implementation Details
  14. EXPERIMENTS
  15. Data Acquisition
  16. ...and 4 more sections

Figures (4)

  • Figure 1: An overview of the presented method for autonomous navigation of a US probe. At each time step, a 2D image is acquired with the current probe pose (a) and serves as the input of the deep Q-network (b). The optimal movement action is selected from 10 actions associated with the 5-DOF pose of the probe (c), and 1 DOF is used to track the patient surface (d). The confidence map (e) is computed from the US image to estimate the image quality, and the reward function (f) encourages the improvement of position, orientation and image quality during the navigation.
  • Figure 2: Illustration of the data acquisition. (a) A robotic US system is used to acquire B-mode US images of the L1-L5 lumbar vertebrae of the volunteers for 3D volume reconstruction. (b) is the visualization of an exemplary data volume in 3D Slicer slicer. (c) illustrates the paramedian sagittal lamina view (PSL) of the spine spineUS, and (d) shows an image of the PSL plane acquired by a clinician.
  • Figure 3: The navigation performance against the image quality improvement of the SonoRL w/o conf agent during training in the intra- and inter-patient settings. As the average confidence improvement per step increases, the average pose improvement per step increases and the final position and orientation errors decrease, showing that the navigation performance is positively correlated with the improvement of image quality.
  • Figure 4: Snapshots of the trajectories of the (a) SonoRL w/o conf and (b) SonoRL w/ conf agents in an intra-patient test case. The 3D plot shows the current plane (green), the goal plane (blue), the patient surface (salmon) and the poses of the current probe and the goal. Below it shows the top-view trajectory of the agent on the $xy$ plane (green), and the goal position is indicated by a red star. The current plane image is displayed on the right side of each 3D plot, with the pose and confidence improvement of the current step marked in green (positive) or red (negative). The confidence map below it shows the pixel-wise confidence in the current image with the average confidence in the ROI (yellow rectangle).