Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Understanding Task Aggregation for Generalizable Ultrasound Foundation Models

Fangyijie Wang, Tanya Akumu, Vien Ngoc Dang, Amelia Jimńez-Sánchez, Jieyun Bai, Guénolé Silvestre, Karim Lekadir, Kathleen M. Curran

Abstract

Foundation models promise to unify multiple clinical tasks within a single framework, but recent ultrasound studies report that unified models can underperform task-specific baselines. We hypothesize that this degradation arises not from model capacity limitations, but from task aggregation strategies that ignore interactions between task heterogeneity and available training data scale. In this work, we systematically analyze when heterogeneous ultrasound tasks can be jointly learned without performance loss, establishing practical criteria for task aggregation in unified clinical imaging models. We introduce M2DINO, a multi-organ, multi-task framework built on DINOv3 with task-conditioned Mixture-of-Experts blocks for adaptive capacity allocation. We systematically evaluate 27 ultrasound tasks spanning segmentation, classification, detection, and regression under three paradigms: task-specific, clinically-grouped, and all-task unified training. Our results show that aggregation effectiveness depends strongly on training data scale. While clinically-grouped training can improve performance in data-rich settings, it may induce substantial negative transfer in low-data settings. In contrast, all-task unified training exhibits more consistent performance across clinical groups. We further observe that task sensitivity varies by task type in our experiments: segmentation shows the largest performance drops compared with regression and classification. These findings provide practical guidance for ultrasound foundation models, emphasizing that aggregation strategies should jointly consider training data availability and task characteristics rather than relying on clinical taxonomy alone.

Understanding Task Aggregation for Generalizable Ultrasound Foundation Models

Abstract

Foundation models promise to unify multiple clinical tasks within a single framework, but recent ultrasound studies report that unified models can underperform task-specific baselines. We hypothesize that this degradation arises not from model capacity limitations, but from task aggregation strategies that ignore interactions between task heterogeneity and available training data scale. In this work, we systematically analyze when heterogeneous ultrasound tasks can be jointly learned without performance loss, establishing practical criteria for task aggregation in unified clinical imaging models. We introduce M2DINO, a multi-organ, multi-task framework built on DINOv3 with task-conditioned Mixture-of-Experts blocks for adaptive capacity allocation. We systematically evaluate 27 ultrasound tasks spanning segmentation, classification, detection, and regression under three paradigms: task-specific, clinically-grouped, and all-task unified training. Our results show that aggregation effectiveness depends strongly on training data scale. While clinically-grouped training can improve performance in data-rich settings, it may induce substantial negative transfer in low-data settings. In contrast, all-task unified training exhibits more consistent performance across clinical groups. We further observe that task sensitivity varies by task type in our experiments: segmentation shows the largest performance drops compared with regression and classification. These findings provide practical guidance for ultrasound foundation models, emphasizing that aggregation strategies should jointly consider training data availability and task characteristics rather than relying on clinical taxonomy alone.
Paper Structure (15 sections, 3 figures, 2 tables)

This paper contains 15 sections, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. Problem Setting and Training Paradigms
  4. DINO Backbone
  5. Mixture of Experts with Task-Conditioned Routing
  6. Task-Specific Heads and Multi-Task Learning
  7. Experiments
  8. Dataset.
  9. Implementation Details.
  10. Evaluation Metrics.
  11. Results
  12. Discussion
  13. Conclusion

Figures (3)

  • Figure 1: Overview of our M2DINO framework. (a) Ultrasound images are processed by a shared DINOv3 encoder augmented with task-conditioned MoE blocks. The unified representation is optimized for segmentation, detection, regression, and classification via task-specific prediction heads. Frozen and trainable components are indicated. (b) A conceptual comparison of the three training paradigms. Although the architecture remains the same, TS, CG, and AU differ in how tasks are aggregated during training and in whether the MoE is enabled.
  • Figure 2: Absolute performance of TS, CG, and AU training paradigms across representative tasks: segmentation (DSC $\uparrow$), classification (AUC $\uparrow$), and regression (MRE $\downarrow$). Abd: Abdomen; MO: Multi-organ.
  • Figure 3: Relative performance change ($\Delta$, %) with respect to TS.