SeCo-INR: Semantically Conditioned Implicit Neural Representations for Improved Medical Image Super-Resolution

TL;DR

This work proposes a novel framework, referred to as the Semantically Conditioned INR (SeCo-INR), that conditions an INR using local priors from a medical image, enabling accurate model fitting and interpolation capabilities to achieve super-resolution.

Abstract

Implicit Neural Representations (INRs) have recently advanced the field of deep learning due to their ability to learn continuous representations of signals without the need for large training datasets. Although INR methods have been studied for medical image super-resolution, their adaptability to localized priors in medical images has not been extensively explored. Medical images contain rich anatomical divisions that could provide valuable local prior information to enhance the accuracy and robustness of INRs. In this work, we propose a novel framework, referred to as the Semantically Conditioned INR (SeCo-INR), that conditions an INR using local priors from a medical image, enabling accurate model fitting and interpolation capabilities to achieve super-resolution. Our framework learns a continuous representation of the semantic segmentation features of a medical image and utilizes it to derive the optimal INR for each semantic region of the image. We tested our framework using several medical imaging modalities and achieved higher quantitative scores and more realistic super-resolution outputs compared to state-of-the-art methods.

Figures (5)

• Figure 1: Illustration of SeCo-INR: Semantically Conditioned Implicit Neural Representations framework for Medical Image Super-Resolution. The Pixel Class Representation Network learns a continuous representation of the semantic segmentation mask, which provides conditioning to the Adapting SIREN Network via the Conditioner Network. This process enables the learning of a more accurate implicit neural representation of the underlying medical image, allowing robust super-resolution capabilities.
• Figure 2: Qualitative evaluation between different INR methods for the Abdominal CT data at resolution factors of $2\times$, $3\times$, and $4\times$. SeCo-INR produces realistic outputs of abdominal organs with sharp edges, less noise, and less blurring.
• Figure 3: Qualitative evaluation between different INR methods for the fastMRI brain data at resolution factors of $1.5\times$, $2\times$, and $2.5\times$. SeCo-INR produces realistic outputs of the brain with intricate anatomical details preserved (see yellow arrow), and less noise and blurring.
• Figure 4: Qualitative evaluation between different INR methods for the BraTS brain MRI data at resolution factors of $1.5\times$ and $2\times$. SeCo-INR produces realistic outputs of the tumor regions with less noise and blurring.
• Figure 5: An example from the fastMRI brain dataset shows the evolution of the segmentation mask learning process using the Pixel Class Representation Network.