Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Multi-modal Medical Image Fusion For Non-Small Cell Lung Cancer Classification

Salma Hassan, Hamad Al Hammadi, Ibrahim Mohammed, Muhammad Haris Khan

TL;DR

An innovative integration of multi-modal data is introduced, synthesizing fused medical imaging with clinical health records and genomic data, which has the potential to transform NSCLC diagnostics, facilitating earlier detection and more effective treatment planning and, ultimately, leading to superior patient outcomes in lung cancer care.

Abstract

The early detection and nuanced subtype classification of non-small cell lung cancer (NSCLC), a predominant cause of cancer mortality worldwide, is a critical and complex issue. In this paper, we introduce an innovative integration of multi-modal data, synthesizing fused medical imaging (CT and PET scans) with clinical health records and genomic data. This unique fusion methodology leverages advanced machine learning models, notably MedClip and BEiT, for sophisticated image feature extraction, setting a new standard in computational oncology. Our research surpasses existing approaches, as evidenced by a substantial enhancement in NSCLC detection and classification precision. The results showcase notable improvements across key performance metrics, including accuracy, precision, recall, and F1-score. Specifically, our leading multi-modal classifier model records an impressive accuracy of 94.04%. We believe that our approach has the potential to transform NSCLC diagnostics, facilitating earlier detection and more effective treatment planning and, ultimately, leading to superior patient outcomes in lung cancer care.

Multi-modal Medical Image Fusion For Non-Small Cell Lung Cancer Classification

TL;DR

An innovative integration of multi-modal data is introduced, synthesizing fused medical imaging with clinical health records and genomic data, which has the potential to transform NSCLC diagnostics, facilitating earlier detection and more effective treatment planning and, ultimately, leading to superior patient outcomes in lung cancer care.

Abstract

The early detection and nuanced subtype classification of non-small cell lung cancer (NSCLC), a predominant cause of cancer mortality worldwide, is a critical and complex issue. In this paper, we introduce an innovative integration of multi-modal data, synthesizing fused medical imaging (CT and PET scans) with clinical health records and genomic data. This unique fusion methodology leverages advanced machine learning models, notably MedClip and BEiT, for sophisticated image feature extraction, setting a new standard in computational oncology. Our research surpasses existing approaches, as evidenced by a substantial enhancement in NSCLC detection and classification precision. The results showcase notable improvements across key performance metrics, including accuracy, precision, recall, and F1-score. Specifically, our leading multi-modal classifier model records an impressive accuracy of 94.04%. We believe that our approach has the potential to transform NSCLC diagnostics, facilitating earlier detection and more effective treatment planning and, ultimately, leading to superior patient outcomes in lung cancer care.
Paper Structure (18 sections, 3 figures, 1 table)

This paper contains 18 sections, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. Proposed Methodology
  4. Comprehensive Pre-processing
  5. Advanced Image Fusion
  6. Baseline Models
  7. Multi-modal Classification Model
  8. Experimental Details
  9. Datasets
  10. Implementation Detail and Result Analysis
  11. Model Training
  12. Evaluation Metrics
  13. Results and Discussion
  14. Quantitative Analysis
  15. Qualitative Analysis
  16. ...and 3 more sections

Figures (3)

  • Figure 1: Multi-modal BEiT model showing model architecture that starts with pre-processing the CT and PET scan and then fusing them into a single scan that is then fused with the other data modalities, such as clinical and genetic, before applying feature selection and passing it to the BEiT model for classification.
  • Figure 2: Example of CT, PET images, and resulting fused image
  • Figure 3: F1-score comparison of different models using fused PET/CT images versus CT images alone.