MedFuncta: A Unified Framework for Learning Efficient Medical Neural Fields

TL;DR

This work introduces MedFuncta, a unified framework for large-scale NF training on diverse medical signals, and introduces a scalable meta-learning strategy for shared network learning that employs sparse supervision during training, thereby reducing memory consumption and computational overhead while maintaining competitive performance.

Abstract

Research in medical imaging primarily focuses on discrete data representations that poorly scale with grid resolution and fail to capture the often continuous nature of the underlying signal. Neural Fields (NFs) offer a powerful alternative by modeling data as continuous functions. While single-instance NFs have successfully been applied in medical contexts, extending them to large-scale medical datasets remains an open challenge. We therefore introduce MedFuncta, a unified framework for large-scale NF training on diverse medical signals. Building on Functa, our approach encodes data into a unified representation, namely a 1D latent vector, that modulates a shared, meta-learned NF, enabling generalization across a dataset. We revisit common design choices, introducing a non-constant frequency parameter $ω$ in widely used SIREN activations, and establish a connection between this $ω$-schedule and layer-wise learning rates, relating our findings to recent work in theoretical learning dynamics. We additionally introduce a scalable meta-learning strategy for shared network learning that employs sparse supervision during training, thereby reducing memory consumption and computational overhead while maintaining competitive performance. Finally, we evaluate MedFuncta across a diverse range of medical datasets and show how to solve relevant downstream tasks on our neural data representation. To promote further research in this direction, we release our code, model weights and the first large-scale dataset - MedNF - containing > 500 k latent vectors for multi-instance medical NFs.

Figures (19)

• Figure 1: (Left) The proposed network with shared parameters $\theta$, that is conditioned by a single signal-specific parameter vector $\phi^{(i)}$. (Right) The proposed meta-learning strategy that, starting from a random initialization of $\theta$, learns shared network parameters $\theta^{*}$ in a way that we can fit a signal by updating $\phi^{(i)}$ for few steps.
• Figure 2: (Left) The proposed approach for meta-learning the shared model parameters $\theta$. An Algorithm describing the full meta-learning approach can be found in \ref{['supp:secord']}. (Right) The proposed test time adaptation scheme.
• Figure 3: Input and reconstruction examples from the hold-out test set for (from left to right)Chest X-ray, Pneumonia Chest X-ray, Retinal OCT, Fundus Camera, Dermatoscope, Colon Histopathology, and Cell Microscopy images.
• Figure 4: Grid search over different $\omega_1$ and $\delta$ parameters. We report $(a)$ PSNR, $(b)$ SSIM and $(c)$ LPIPS after 25k iterations. Outliers with red borders were excluded from color scaling. Measured on the Chest X-ray dataset$(64 \times 64)$.
• Figure 5: Development of $(a)$ PSNR, $(b)$ MSE, $(c)$ SSIM, and $(d)$ LPIPS on the validation set throughout a training run ($250k$ iterations).