Adaptive Fusion of Radiomics and Deep Features for Lung Adenocarcinoma Subtype Recognition

TL;DR

This study tackles LUAD subtype recognition by jointly modeling pre-invasive and invasive categories and distinguishing IA subtypes through CT-based nodules. It introduces MHA-FF, an attention-based fusion of hand-crafted radiomics and deep learning features, with SIS feature selection to reduce radiomics dimensionality. On a multicenter dataset, the method achieves about $0.906$ AUC for Pre-IA vs IA and around $0.740$–$0.873$ task-specific accuracy for IA subtypes, outperforming radiomics-only and early fusion baselines. The approach holds promise for supporting surgical decision-making and personalized treatment planning, though external validation and robustness to slice thickness remain as future work.

Abstract

The most common type of lung cancer, lung adenocarcinoma (LUAD), has been increasingly detected since the advent of low-dose computed tomography screening technology. In clinical practice, pre-invasive LUAD (Pre-IAs) should only require regular follow-up care, while invasive LUAD (IAs) should receive immediate treatment with appropriate lung cancer resection, based on the cancer subtype. However, prior research on diagnosing LUAD has mainly focused on classifying Pre-IAs/IAs, as techniques for distinguishing different subtypes of IAs have been lacking. In this study, we proposed a multi-head attentional feature fusion (MHA-FF) model for not only distinguishing IAs from Pre-IAs, but also for distinguishing the different subtypes of IAs. To predict the subtype of each nodule accurately, we leveraged both radiomics and deep features extracted from computed tomography images. Furthermore, those features were aggregated through an adaptive fusion module that can learn attention-based discriminative features. The utility of our proposed method is demonstrated here by means of real-world data collected from a multi-center cohort.

Figures (5)

• Figure 1: Illustrations of different subtypes of LUAD. The top panel shows some examples of Pre-IA, and the bottom panel shows the subtypes of IA. Specifically, recognizing the subtypes of IA would be helpful for determining an appropriate surgical model 2020Procedure.
• Figure 2: Conceptual diagram of the proposed method for LUAD subtype recognition. The input is an array of CT images with regard to a specific patient, with nodule centers manually labelled, in advance. The input is first passed through two modules (A and B), in parallel. Module A involves knowledge-driven radiomics feature extraction, while Module B involves data-driven deep feature extraction. For module A, we obtained both the lung mask and multi-scale nodule mask to extract various radiomics features. A feature-selection mechanism (i.e., a sure independence screening (SIS) feature-selection mechanism) is designed to reveal the radiomics features with a significant impact on the prediction, i.e., $x^r$. For module B, a cropped image $I_{N_{k}}$ at the nodule center is fed into a 2D-convoluted neural network (CNN) to extract a deep semantic feature $x^d$ for each slice. Next, both the radiomics and deep features are fed into a multi-head attentional block (i.e., module C) for feature fusion. The final probabilistic prediction of LUAD subtype (i.e., HDA, MDA, and PDA) is obtained by mean pooling plus softmax activation. It should be noted that the output will be changed to a binary indicator when the task becomes distinguishing IAs from Pre-IAs.
• Figure 3: Averaged attentional weights of radiomics features for each head in MHA-FF$\times$4.
• Figure 4: Per-category average attentional weights of radiomics and deep features.
• Figure 5: Grad-cam feature map of HDA,MDA, and PDA nodules.