FACT: Foundation Model for Assessing Cancer Tissue Margins with Mass Spectrometry

TL;DR

The paper tackles real-time cancer tissue-margin classification using REIMS data in settings with limited labeled examples. It introduces FACT, a CLAP-based foundation model with supervised triplet pretraining that learns discriminative spectral embeddings, enabling accurate cancer vs benign classification with AUROC $82.4\% \pm 0.8$ and 72 ms inference per spectrum. The study demonstrates that the CLAP backbone combined with triplet loss yields robust performance, outperforming several baselines and several self-supervised/semi-supervised approaches, and provides insight into failure modes due to data quality. The findings support the viability of domain-adapted foundation models for intraoperative decision support in data-scarce clinical environments and outline concrete avenues for future extensions, including multi-modal prompts and broader spectral tasks.

Abstract

Purpose: Accurately classifying tissue margins during cancer surgeries is crucial for ensuring complete tumor removal. Rapid Evaporative Ionization Mass Spectrometry (REIMS), a tool for real-time intraoperative margin assessment, generates spectra that require machine learning models to support clinical decision-making. However, the scarcity of labeled data in surgical contexts presents a significant challenge. This study is the first to develop a foundation model tailored specifically for REIMS data, addressing this limitation and advancing real-time surgical margin assessment. Methods: We propose FACT, a Foundation model for Assessing Cancer Tissue margins. FACT is an adaptation of a foundation model originally designed for text-audio association, pretrained using our proposed supervised contrastive approach based on triplet loss. An ablation study is performed to compare our proposed model against other models and pretraining methods. Results: Our proposed model significantly improves the classification performance, achieving state-of-the-art performance with an AUROC of $82.4\% \pm 0.8$. The results demonstrate the advantage of our proposed pretraining method and selected backbone over the self-supervised and semi-supervised baselines and alternative models. Conclusion: Our findings demonstrate that foundation models, adapted and pretrained using our novel approach, can effectively classify REIMS data even with limited labeled examples. This highlights the viability of foundation models for enhancing real-time surgical margin assessment, particularly in data-scarce clinical environments.

Figures (3)

• Figure 1: Overview of the FACT architecture and training process. The input spectra are tokenized, projected, and passed through a Swin Transformer encoder adapted from CLAP via transfer learning. The encoded spectrum embeddings are projected and classified using dedicated MLP heads. The model is trained in two distinct steps: pretraining using triplet loss, and finetuning using cross entropy loss.
• Figure 2: (a) The embedding space of FACT reduced to 2 dimensions using UMAP. Decision boundaries of the classes are visualized as contours using the training samples, while test samples are displayed as points, with crosses depicting negative samples and circles designating positive samples. Samples are colored based on how frequently they are correctly classified. Most samples align with the decision boundaries, and are thus, shown in a darker shade, but a few samples, often near the edges, are misclassified. (b) and (c) are sample spectra with high failure rate drawn from the test set, which are labeled positive and negative, respectively.
• Figure 3: Performance of foundation models under different pretraining methods. The figure illustrates the mean and the standard deviation of each metric under 15 runs.