Comprehensive Review of Reinforcement Learning for Medical Ultrasound Imaging

TL;DR

This paper surveys reinforcement learning (RL) applications in medical ultrasound (US) imaging and proposes a taxonomy that maps the four US stages—acquisition, enhancement, analysis, and decision-making—onto a five-stage RL development pipeline (data preparation, problem formulation, simulation, implementation, validation). It systematically reviews RL methods across acquisition, enhancement, and analysis tasks, highlighting how state, action, and reward design, along with simulation environments, enable autonomous US capabilities. Key contributions include a cross-pipeline taxonomy, a synthesis of data sources and validation metrics, and a discussion of sim-to-real transfer, data scarcity, and interpretability challenges. The work underscores the potential of RL to improve US imaging throughput, consistency, and diagnostic support, while outlining concrete opportunities for standardization, multi-agent approaches, and real-world validation to advance clinical adoption.

Abstract

Medical Ultrasound (US) imaging has seen increasing demands over the past years, becoming one of the most preferred imaging modalities in clinical practice due to its affordability, portability, and real-time capabilities. However, it faces several challenges that limit its applicability, such as operator dependency, variability in interpretation, and limited resolution, which are amplified by the low availability of trained experts. This calls for the need of autonomous systems that are capable of reducing the dependency on humans for increased efficiency and throughput. Reinforcement Learning (RL) comes as a rapidly advancing field under Artificial Intelligence (AI) that allows the development of autonomous and intelligent agents that are capable of executing complex tasks through rewarded interactions with their environments. Existing surveys on advancements in the US scanning domain predominantly focus on partially autonomous solutions leveraging AI for scanning guidance, organ identification, plane recognition, and diagnosis. However, none of these surveys explore the intersection between the stages of the US process and the recent advancements in RL solutions. To bridge this gap, this review proposes a comprehensive taxonomy that integrates the stages of the US process with the RL development pipeline. This taxonomy not only highlights recent RL advancements in the US domain but also identifies unresolved challenges crucial for achieving fully autonomous US systems. This work aims to offer a thorough review of current research efforts, highlighting the potential of RL in building autonomous US solutions while identifying limitations and opportunities for further advancements in this field.

Figures (22)

• Figure 2: Survey Outline (Ultrasound (US) Imaging & Reinforcement Learning (RL)).
• Figure 3: Ultrasound Imaging Illustration
• Figure 4: The Different Modalities of Ultrasound (US) Imaging bohilțea2022clinically.
• Figure 5: Taxonomy of Reinforcement Learning (RL) in Ultrasound (US) Imaging.
• Figure 6: Optimizing Reinforcement Learning (RL) Policy with Predefined Rewards and Expert Demonstrations ning2023inverse.