Shifting Focus: From Global Semantics to Local Prominent Features in Swin-Transformer for Knee Osteoarthritis Severity Assessment

TL;DR

This paper tackles knee osteoarthritis grading from radiographs by shifting from global semantic focus to local, task-relevant features. It introduces a Swin Transformer–based framework augmented with a Multi-Prediction Head Network, skip connections, and a Negative Cosine Similarity Loss to align local features with the classifier’s decision distribution, optimizing the joint objective $\mathcal{L} = \sum_{k=1}^{K} \mathcal{L}_{BCE_k} + \lambda \mathcal{L}_{NCSL}$. The method is pretrained on MOST and finetuned on OAI, and ablation studies show MPHN with NCSL provides the best performance, achieving ACC $\approx 72.4\%$ and F1 $\approx 0.704$ on the OAI test set. Qualitative Grad-CAM visualizations confirm the model leverages both local textures and global joint structure across KL grades, and the approach surpasses several state-of-the-art KOA methods. Overall, the work offers a robust, clinically relevant advance toward reliable automated KOA diagnostics with potential impact on radiology workflows.

Abstract

Conventional imaging diagnostics frequently encounter bottlenecks due to manual inspection, which can lead to delays and inconsistencies. Although deep learning offers a pathway to automation and enhanced accuracy, foundational models in computer vision often emphasize global context at the expense of local details, which are vital for medical imaging diagnostics. To address this, we harness the Swin Transformer's capacity to discern extended spatial dependencies within images through the hierarchical framework. Our novel contribution lies in refining local feature representations, orienting them specifically toward the final distribution of the classifier. This method ensures that local features are not only preserved but are also enriched with task-specific information, enhancing their relevance and detail at every hierarchical level. By implementing this strategy, our model demonstrates significant robustness and precision, as evidenced by extensive validation of two established benchmarks for Knee OsteoArthritis (KOA) grade classification. These results highlight our approach's effectiveness and its promising implications for the future of medical imaging diagnostics. Our implementation is available on https://github.com/mtliba/KOA_NLCS2024

Figures (2)

• Figure 1: Overview of the Proposed KOA Diagnostic Model. This figure outlines our advanced model, highlighting the Swin Transformer for hierarchical feature extraction, multiple prediction heads for detailed KL grade classification, skip connections for effective feature flow, and Negative Cosine Similarity Loss for feature optimization. Together, these elements illustrate our novel approach to balancing local detail recognition with global feature abstraction for improved KOA diagnostic accuracy.
• Figure 2: selvaraju2017grad demonstrate our model's ability to discern progression in KOA severity from KL grade 0 (healthy) to KL grade 4 (most severe). These visualizations underscore the model's focus on both local and global features, adjusting according to severity. They confirm our model's effective use of various hierarchical feature levels without central region bias, relaying on both joint' local textures and global form