Privacy-Aware Continual Self-Supervised Learning on Multi-Window Chest Computed Tomography for Domain-Shift Robustness

TL;DR

Domain shifts in chest CT imaging, driven by multi-window settings, complicate robust diagnosis under strict privacy constraints. The authors propose a privacy-aware continual self-supervised learning framework that combines latent replay with a Wasserstein-distance–based knowledge distillation and batch knowledge ensemble (WKD-BKE) to learn domain-robust representations across window settings, formalized through the WKD loss $\mathcal{L}_{\text{WKD}} = \gamma \mathrm{D}_{\text{mean}}(\boldsymbol{\mu}^{\mathcal{T}}, \boldsymbol{\mu}^{\mathcal{S}}) + \mathrm{D}_{\operatorname{cov}}(\boldsymbol{\Sigma}^{\mathcal{T}}, \boldsymbol{\Sigma}^{\mathcal{S}})$ and a batch-based distillation mechanism. The method unfolds in three stages: Stage1 self-supervised pretraining on the first-domain dataset, Stage2 latent feature sampling into a memory buffer to preserve privacy, and Stage3 continual self-supervised learning on the second-domain dataset with feature distillation, followed by fine-tuning on labeled data. Experiments on two public chest-CT datasets show that the approach outperforms state-of-the-art SSL and CSSL baselines, achieving better domain robustness while maintaining data privacy. These results demonstrate a practical, privacy-preserving path to robust medical imaging models that can adapt to evolving clinical imaging conditions.

Abstract

We propose a novel continual self-supervised learning (CSSL) framework for simultaneously learning diverse features from multi-window-obtained chest computed tomography (CT) images and ensuring data privacy. Achieving a robust and highly generalizable model in medical image diagnosis is challenging, mainly because of issues, such as the scarcity of large-scale, accurately annotated datasets and domain shifts inherent to dynamic healthcare environments. Specifically, in chest CT, these domain shifts often arise from differences in window settings, which are optimized for distinct clinical purposes. Previous CSSL frameworks often mitigated domain shift by reusing past data, a typically impractical approach owing to privacy constraints. Our approach addresses these challenges by effectively capturing the relationship between previously learned knowledge and new information across different training stages through continual pretraining on unlabeled images. Specifically, by incorporating a latent replay-based mechanism into CSSL, our method mitigates catastrophic forgetting due to domain shifts during continual pretraining while ensuring data privacy. Additionally, we introduce a feature distillation technique that integrates Wasserstein distance-based knowledge distillation (WKD) and batch-knowledge ensemble (BKE), enhancing the ability of the model to learn meaningful, domain-shift-robust representations. Finally, we validate our approach using chest CT images obtained across two different window settings, demonstrating superior performance compared with other approaches.

Figures (5)

• Figure 1: Overview of the proposed continual self-supervised learning (CSSL) framework.
• Figure 2: Examples of chest CT images on the subsets from the J-MID database: (a) First-domain dataset ($D_1$) and (b) Second-domain dataset ($D_2$).
• Figure 3: Examples of chest CT images on the subset from the RICORD dataset: (a) $D_1$ and (b) $D_2$.
• Figure 4: Examples of chest CT images on the SARS-CoV-2 CT-Scan Dataset: (a) COVID-19 and (b) Normal.
• Figure 5: Examples of chest CT images from the Chest CT-Scan Images dataset: (a) adenocarcinoma, (b) large-cell carcinoma, (c) squamous-cell carcinoma, and (d) normal.