NOIR: Neural Operator mapping for Implicit Representations

Abstract

This paper presents NOIR, a framework that reframes core medical imaging tasks as operator learning between continuous function spaces, challenging the prevailing paradigm of discrete grid-based deep learning. Instead of operating on fixed pixel or voxel grids, NOIR embeds discrete medical signals into shared Implicit Neural Representations and learns a Neural Operator that maps between their latent modulations, enabling resolution-independent function-to-function transformations. We evaluate NOIR across multiple 2D and 3D downstream tasks, including segmentation, shape completion, image-to-image translation, and image synthesis, on several public datasets such as Shenzhen, OASIS-4, SkullBreak, fastMRI, as well as an in-house clinical dataset. It achieves competitive performance at native resolution while demonstrating strong robustness to unseen discretizations, and empirically satisfies key theoretical properties of neural operators. The project page is available here: https://github.com/Sidaty1/NOIR-io.

Figures (11)

• Figure 1: Downstream tasks solved by NOIR. By formulating each task as operator learning between function spaces, NOIR addresses segmentation, completion, synthesis, and translation across in 2D and 3D, with inherent robustness to resolution changes.
• Figure 2: Architecture of NOIR. Discrete input ($f^d_{i}$) and output ($g^d_{i}$) signals are embedded as continuous functions ($f_{i|\mathcal{X}}$ and $g_{i|\mathcal{X}}$) using INRs with shared dataset-level parameters and signal-specific modulations. A neural operator maps the modulations of the input ($z_{in}$) and output ($z_{out}$) INRs, resulting in a continuous function-to-function mapping applicable to multiple downstream tasks.
• Figure 3: Qualitative results for segmentation and anatomy completion. Rows correspond to Shenzhen binary segmentation, OASIS-4 multi-label segmentation, and SkullBreak anatomy completion, respectively. A randomly selected test sample is shown with the input image, ground truth, NOIR results, and comparison methods.
• Figure 4: Qualitative results for image synthesis (top row) and image translation (bottom row). A randomly selected test sample is shown, including the input image, ground truth, NOIR results, and results from comparison methods.
• Figure 5: Resolution robustness of NOIR. (a) Dice distributions across discretizations show stable performance with mean Dice around $0.93$--$0.94$. (b) Qualitative example demonstrating consistent segmentation predictions across different input resolutions.