Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learning to learn skill assessment for fetal ultrasound scanning

Yipei Wang, Qianye Yang, Lior Drukker, Aris T. Papageorghiou, Yipeng Hu, J. Alison Noble

TL;DR

The paper tackles the challenge of subjective fetal ultrasound skill assessment by introducing a bi-level optimisation framework that jointly learns a task predictor for a segmentation-based clinical task and a skill predictor that gauges frame sequences by how well the task predictor performs. Skill scores weight the task loss and vice versa, enabling task-centric, frame-level skill evaluation without manually defined ratings. Validated on the PULSE fetal ultrasound dataset with three landmarks, the approach demonstrates that frames deemed high-skill by the predictor tend to enhance downstream task performance, supporting real-time, task-specific guidance for sonographers. The method offers a scalable, label-efficient alternative to conventional supervision, with potential for adaptation to other clinical tasks and prospective clinical validation.

Abstract

Traditionally, ultrasound skill assessment has relied on expert supervision and feedback, a process known for its subjectivity and time-intensive nature. Previous works on quantitative and automated skill assessment have predominantly employed supervised learning methods, often limiting the analysis to predetermined or assumed factors considered influential in determining skill levels. In this work, we propose a novel bi-level optimisation framework that assesses fetal ultrasound skills by how well a task is performed on the acquired fetal ultrasound images, without using manually predefined skill ratings. The framework consists of a clinical task predictor and a skill predictor, which are optimised jointly by refining the two networks simultaneously. We validate the proposed method on real-world clinical ultrasound videos of scanning the fetal head. The results demonstrate the feasibility of predicting ultrasound skills by the proposed framework, which quantifies optimised task performance as a skill indicator.

Learning to learn skill assessment for fetal ultrasound scanning

TL;DR

The paper tackles the challenge of subjective fetal ultrasound skill assessment by introducing a bi-level optimisation framework that jointly learns a task predictor for a segmentation-based clinical task and a skill predictor that gauges frame sequences by how well the task predictor performs. Skill scores weight the task loss and vice versa, enabling task-centric, frame-level skill evaluation without manually defined ratings. Validated on the PULSE fetal ultrasound dataset with three landmarks, the approach demonstrates that frames deemed high-skill by the predictor tend to enhance downstream task performance, supporting real-time, task-specific guidance for sonographers. The method offers a scalable, label-efficient alternative to conventional supervision, with potential for adaptation to other clinical tasks and prospective clinical validation.

Abstract

Traditionally, ultrasound skill assessment has relied on expert supervision and feedback, a process known for its subjectivity and time-intensive nature. Previous works on quantitative and automated skill assessment have predominantly employed supervised learning methods, often limiting the analysis to predetermined or assumed factors considered influential in determining skill levels. In this work, we propose a novel bi-level optimisation framework that assesses fetal ultrasound skills by how well a task is performed on the acquired fetal ultrasound images, without using manually predefined skill ratings. The framework consists of a clinical task predictor and a skill predictor, which are optimised jointly by refining the two networks simultaneously. We validate the proposed method on real-world clinical ultrasound videos of scanning the fetal head. The results demonstrate the feasibility of predicting ultrasound skills by the proposed framework, which quantifies optimised task performance as a skill indicator.
Paper Structure (35 sections, 9 equations, 7 figures, 9 tables, 1 algorithm)

This paper contains 35 sections, 9 equations, 7 figures, 9 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Method
  3. Task predictor
  4. Skill predictor
  5. Optimising the task predictor
  6. Optimising the skill predictor
  7. Bi-level optimisation of the networks
  8. Experiments
  9. Model selection
  10. Direct evaluation
  11. Meta learning evaluation
  12. Meta evaluation dataset
  13. Fine-Tuning on meta-evaluation train dataset
  14. Testing on meta-evaluation test dataset
  15. Clinical task and dataset curation
  16. ...and 20 more sections

Figures (7)

  • Figure 1: Overview of the proposed method.
  • Figure 2: Illustration of the proposed bi-level optimisation framework for ultrasound skill assessment.
  • Figure 3: Example frames with and without annotations for the anatomical landmarks.
  • Figure 4: Meta evaluation results of the task predictor. Solid lines with different colours represent model performance fine-tuned using different ratios of the test dataset, while the shades indicate the variance.
  • Figure 5: Meta evaluation results of the skill predictor. Different colours represent models fine-tuned using different ratios of the test dataset.
  • ...and 2 more figures