Knee Osteoarthritis Severity Prediction using an Attentive Multi-Scale Deep Convolutional Neural Network

TL;DR

A deep learning-based framework that automatically assesses the Knee OA severity in terms of Kellgren and Lawrence (KL) grade classification from X-rays, built upon one of the most recent deep models, called the High-Resolution Network (HRNet), to capture the multi-scale features of knee X-rays.

Abstract

Knee Osteoarthritis (OA) is a destructive joint disease identified by joint stiffness, pain, and functional disability concerning millions of lives across the globe. It is generally assessed by evaluating physical symptoms, medical history, and other joint screening tests like radiographs, Magnetic Resonance Imaging (MRI), and Computed Tomography (CT) scans. Unfortunately, the conventional methods are very subjective, which forms a barrier in detecting the disease progression at an early stage. This paper presents a deep learning-based framework, namely OsteoHRNet, that automatically assesses the Knee OA severity in terms of Kellgren and Lawrence (KL) grade classification from X-rays. As a primary novelty, the proposed approach is built upon one of the most recent deep models, called the High-Resolution Network (HRNet), to capture the multi-scale features of knee X-rays. In addition, we have also incorporated an attention mechanism to filter out the counterproductive features and boost the performance further. Our proposed model has achieved the best multiclass accuracy of 71.74% and MAE of 0.311 on the baseline cohort of the OAI dataset, which is a remarkable gain over the existing best-published works. We have also employed the Gradient-based Class Activation Maps (Grad-CAMs) visualization to justify the proposed network learning.

Figures (14)

• Figure 1: Knee OA disease progression: A qualitative demonstration of sample X-rays and their corresponding KL grades.
• Figure 2: An overview of the architecture of the proposed OsteoHRNet for the knee OA severity prediction. Blocks with different colors denote convolution features at different spatial scales. The proposed model takes knee X-Ray image as input and estimates the OA severity in terms of KL grade.
• Figure 3: Connections in HRNet: (a) Multi Resolution convolution in parallel, (b) Fusion of Multi-Resolution convolution. Different colors denote feature resolution at various scales.
• Figure 4: Graphical demonstration of how HRNet fuses information from different resolutions.
• Figure 5: Confusion matrices for KL grade prediction using different competing approaches chen2019fully, Yong and OsteoHRNet.