Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Coaching a Robotic Sonographer: Learning Robotic Ultrasound with Sparse Expert's Feedback

Deepak Raina, Mythra V. Balakuntala, Byung Wook Kim, Juan Wachs, Richard Voyles

TL;DR

A coaching framework for RUS to amplify its performance is proposed, which combines DRL (self-supervised practice) with sparse expert’s feedback through coaching and is modeled as a Partially Observable Markov Decision Process (POMDP), which updates the policy parameters based on the correction by the expert.

Abstract

Ultrasound is widely employed for clinical intervention and diagnosis, due to its advantages of offering non-invasive, radiation-free, and real-time imaging. However, the accessibility of this dexterous procedure is limited due to the substantial training and expertise required of operators. The robotic ultrasound (RUS) offers a viable solution to address this limitation; nonetheless, achieving human-level proficiency remains challenging. Learning from demonstrations (LfD) methods have been explored in RUS, which learns the policy prior from a dataset of offline demonstrations to encode the mental model of the expert sonographer. However, active engagement of experts, i.e. Coaching, during the training of RUS has not been explored thus far. Coaching is known for enhancing efficiency and performance in human training. This paper proposes a coaching framework for RUS to amplify its performance. The framework combines DRL (self-supervised practice) with sparse expert's feedback through coaching. The DRL employs an off-policy Soft Actor-Critic (SAC) network, with a reward based on image quality rating. The coaching by experts is modeled as a Partially Observable Markov Decision Process (POMDP), which updates the policy parameters based on the correction by the expert. The validation study on phantoms showed that coaching increases the learning rate by $25\%$ and the number of high-quality image acquisition by $74.5\%$.

Coaching a Robotic Sonographer: Learning Robotic Ultrasound with Sparse Expert's Feedback

TL;DR

A coaching framework for RUS to amplify its performance is proposed, which combines DRL (self-supervised practice) with sparse expert’s feedback through coaching and is modeled as a Partially Observable Markov Decision Process (POMDP), which updates the policy parameters based on the correction by the expert.

Abstract

Ultrasound is widely employed for clinical intervention and diagnosis, due to its advantages of offering non-invasive, radiation-free, and real-time imaging. However, the accessibility of this dexterous procedure is limited due to the substantial training and expertise required of operators. The robotic ultrasound (RUS) offers a viable solution to address this limitation; nonetheless, achieving human-level proficiency remains challenging. Learning from demonstrations (LfD) methods have been explored in RUS, which learns the policy prior from a dataset of offline demonstrations to encode the mental model of the expert sonographer. However, active engagement of experts, i.e. Coaching, during the training of RUS has not been explored thus far. Coaching is known for enhancing efficiency and performance in human training. This paper proposes a coaching framework for RUS to amplify its performance. The framework combines DRL (self-supervised practice) with sparse expert's feedback through coaching. The DRL employs an off-policy Soft Actor-Critic (SAC) network, with a reward based on image quality rating. The coaching by experts is modeled as a Partially Observable Markov Decision Process (POMDP), which updates the policy parameters based on the correction by the expert. The validation study on phantoms showed that coaching increases the learning rate by and the number of high-quality image acquisition by .
Paper Structure (18 sections, 7 equations, 5 figures, 1 table, 1 algorithm)

This paper contains 18 sections, 7 equations, 5 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related work
  3. Robotic ultrasound systems
  4. Experts' engagement in learning of RUS
  5. Coaching robots
  6. Methodology
  7. Self-supervised practice through DRL
  8. State space
  9. Action space
  10. Rewards
  11. Coaching with sparse expert's feedback
  12. POMDP model
  13. Approximate solution
  14. Results and Discussions
  15. Experimental setup
  16. ...and 3 more sections

Figures (5)

  • Figure 1: Improving the learning rate and performance of DRL policy for robotic ultrasound procedure with sparse coaching by expert sonographers.
  • Figure 2: State space representation using a deep convolution neural network
  • Figure 3: The correction of trajectory $\Pi^r$ based on coach's correction to obtain preferred trajectory $\Pi^c$
  • Figure 4: (a) Robotic ultrasound system (b) Urinary bladder phantom (P1) for testing (c) Acquired high-quality ultrasound image ($q=5$) from phantom P0
  • Figure 5: Comparison of average reward over training timesteps for policy learning with and without coaching