FUSECAPS: Investigating Feature Fusion Based Framework for Capsule Endoscopy Image Classification

TL;DR

A hybrid feature extraction method that combines convolutional neural networks, multi-layer perceptrons, and radiomics is suggested, which captures both deep and handmade representations and produces a model with higher generalization and accuracy.

Abstract

In order to improve model accuracy, generalization, and class imbalance issues, this work offers a strong methodology for classifying endoscopic images. We suggest a hybrid feature extraction method that combines convolutional neural networks (CNNs), multi-layer perceptrons (MLPs), and radiomics. Rich, multi-scale feature extraction is made possible by this combination, which captures both deep and handmade representations. These features are then used by a classification head to classify diseases, producing a model with higher generalization and accuracy. In this framework we have achieved a validation accuracy of 76.2% in the capsule endoscopy video frame classification task.

Figures (3)

• Figure 1: Block diagram of the developed pipeline.
• Figure 2: Training and Validation Loss and Accuracy Over Epochs
• Figure 3: AUC-ROC Curve