BackMix: Mitigating Shortcut Learning in Echocardiography with Minimal Supervision

TL;DR

This paper addresses shortcut learning in echocardiography view classification caused by background metadata correlations. It introduces BackMix, a simple background augmentation that swaps ultrasound sector content with backgrounds from other training images to force models to rely on intra-sector information. The approach is extended to a semi-supervised setting with a fraction $f$ of segmentation masks and a loss weighting scheme $w_{BackMix}$ controlled by $\lambda$, enabling strong gains even with as little as $f=0.05$. On in-distribution TMED and out-of-distribution WASE Normals data, BackMix improves accuracy and attention metrics, as measured by energy percentage $\%E$ and focus percentage $\%F$, indicating better generalisability without inference-time background removal. These findings suggest practical improvements for robust echocardiography analysis and point to future work in feature-space augmentation.

Abstract

Neural networks can learn spurious correlations that lead to the correct prediction in a validation set, but generalise poorly because the predictions are right for the wrong reason. This undesired learning of naive shortcuts (Clever Hans effect) can happen for example in echocardiogram view classification when background cues (e.g. metadata) are biased towards a class and the model learns to focus on those background features instead of on the image content. We propose a simple, yet effective random background augmentation method called BackMix, which samples random backgrounds from other examples in the training set. By enforcing the background to be uncorrelated with the outcome, the model learns to focus on the data within the ultrasound sector and becomes invariant to the regions outside this. We extend our method in a semi-supervised setting, finding that the positive effects of BackMix are maintained with as few as 5% of segmentation labels. A loss weighting mechanism, wBackMix, is also proposed to increase the contribution of the augmented examples. We validate our method on both in-distribution and out-of-distribution datasets, demonstrating significant improvements in classification accuracy, region focus and generalisability. Our source code is available at: https://github.com/kitbransby/BackMix

Figures (5)

• Figure 1: Examples of echocardiograms: Unwanted text and measurement data can be seen in regions outside the ultrasound sector. View label superimposed on bottom left.
• Figure 2: Examples of shortcut learning in TMED dataset induced by metadata, which can be specific to the manufacturer, software and operator, but not the patient.
• Figure 3: Training schematic: Backgrounds are shuffled between a subset of training examples, and the prediction loss re-weighted in favour of these examples
• Figure 4: Qualitative results on TMED with GradCAM heatmaps.
• Figure 5: Qualitative results on WASE with GradCAM heatmaps.