SLAM-AGS: Slide-Label Aware Multi-Task Pretraining Using Adaptive Gradient Surgery in Computational Cytology
Marco Acerbis, Swarnadip Chatterjee, Christophe Avenel, Joakim Lindblad
TL;DR
This work tackles learning from weak slide-level labels in computational cytology where witness rates are extremely low. It introduces SLAM-AGS, a slide-label aware multitask pretraining framework that combines a Similarity loss for slide-negative patches and a SimCLR-based contrastive loss for slide-positive patches, augmented by Adaptive Gradient Surgery to mitigate gradient conflicts. On a bone-marrow cytology dataset with simulated witness rates, SLAM-AGS yields superior bag-level F1 and Top-400 key-patch retrieval relative to weakly supervised and self-supervised baselines, with the largest gains at the lowest witness rates. The approach is released openly to facilitate reproducibility and demonstrates that managing gradient interference is crucial for stable, label-efficient pretraining in multi-task MIL settings.
Abstract
Computational cytology faces two major challenges: i) instance-level labels are unreliable and prohibitively costly to obtain, ii) witness rates are extremely low. We propose SLAM-AGS, a Slide-Label-Aware Multitask pretraining framework that jointly optimizes (i) a weakly supervised similarity objective on slide-negative patches and (ii) a self-supervised contrastive objective on slide-positive patches, yielding stronger performance on downstream tasks. To stabilize learning, we apply Adaptive Gradient Surgery to tackle conflicting task gradients and prevent model collapse. We integrate the pretrained encoder into an attention-based Multiple Instance Learning aggregator for bag-level prediction and attention-guided retrieval of the most abnormal instances in a bag. On a publicly available bone-marrow cytology dataset, with simulated witness rates from 10% down to 0.5%, SLAM-AGS improves bag-level F1-Score and Top 400 positive cell retrieval over other pretraining methods, with the largest gains at low witness rates, showing that resolving gradient interference enables stable pretraining and better performance on downstream tasks. To facilitate reproducibility, we share our complete implementation and evaluation framework as open source: https://github.com/Ace95/SLAM-AGS.