MedDet: Generative Adversarial Distillation for Efficient Cervical Disc Herniation Detection

TL;DR

This work introduced MedDet, which leverages the multi-teacher single-student knowledge distillation for model compression and efficiency, meanwhile integrating generative adversarial training to enhance performance, and customize the second-order nmODE to improve the model’s resistance to noise in MRI.

Abstract

Cervical disc herniation (CDH) is a prevalent musculoskeletal disorder that significantly impacts health and requires labor-intensive analysis from experts. Despite advancements in automated detection of medical imaging, two significant challenges hinder the real-world application of these methods. First, the computational complexity and resource demands present a significant gap for real-time application. Second, noise in MRI reduces the effectiveness of existing methods by distorting feature extraction. To address these challenges, we propose three key contributions: Firstly, we introduced MedDet, which leverages the multi-teacher single-student knowledge distillation for model compression and efficiency, meanwhile integrating generative adversarial training to enhance performance. Additionally, we customize the second-order nmODE to improve the model's resistance to noise in MRI. Lastly, we conducted comprehensive experiments on the CDH-1848 dataset, achieving up to a 5% improvement in mAP compared to previous methods. Our approach also delivers over 5 times faster inference speed, with approximately 67.8% reduction in parameters and 36.9% reduction in FLOPs compared to the teacher model. These advancements significantly enhance the performance and efficiency of automated CDH detection, demonstrating promising potential for future application in clinical practice. See project website https://steve-zeyu-zhang.github.io/MedDet

Figures (7)

• Figure 1: The left subfigure illustrates the overall architecture of our proposed MedDet model, which includes a novel adversarial auxiliary teacher module (AATM) for generative adversarial distillation, an adaptive feature alignment (AFA), and a learnable weighted feature fusion (LWFF) module to fuse features from teacher networks and align them with those of the student network. Additionally, it incorporates a denoising nmODE$^2$ module in the detection head. The right subfigure shows that our method achieves superior efficiency compared to teacher models in terms of FLOPs, parameter count, and inference speed.
• Figure 2: The diagram of the adversarial auxiliary teacher module (AATM). The $F_{T}^{i}$ and $F_{S}^{i}$ represent the i-th feature maps outputted by the teacher network and the student network's FPN respectively. G represents the generator.
• Figure 3: The diagram illustrates the architecture of the nmODE$^2$ block, integrated into the classification and regression heads of the teacher networks.
• Figure 4: The figure illustrates the adaptive feature alignment (AFA). Subfigure (a) shows the channel-wise alignment model, while subfigure (b) shows the height-width (HW) alignment model.
• Figure 5: The figure illustrates the learnable weighted feature fusion (LWFF) module, where $M$ denotes the multiplication operation and $C$ denotes the concatenation operation.