MIP Candy: A Modular PyTorch Framework for Medical Image Processing

TL;DR

MIP Candy is presented, a freely available, PyTorch-based framework designed specifically for medical image processing that provides a complete, modular pipeline spanning data loading, training, inference, and evaluation, allowing researchers to obtain a fully functional process workflow by implementing a single method.

Abstract

Medical image processing demands specialized software that handles high-dimensional volumetric data, heterogeneous file formats, and domain-specific training procedures. Existing frameworks either provide low-level components that require substantial integration effort or impose rigid, monolithic pipelines that resist modification. We present MIP Candy (MIPCandy), a freely available, PyTorch-based framework designed specifically for medical image processing. MIPCandy provides a complete, modular pipeline spanning data loading, training, inference, and evaluation, allowing researchers to obtain a fully functional process workflow by implementing a single method, $\texttt{build_network}$, while retaining fine-grained control over every component. Central to the design is $\texttt{LayerT}$, a deferred configuration mechanism that enables runtime substitution of convolution, normalization, and activation modules without subclassing. The framework further offers built-in $k$-fold cross-validation, dataset inspection with automatic region-of-interest detection, deep supervision, exponential moving average, multi-frontend experiment tracking (Weights & Biases, Notion, MLflow), training state recovery, and validation score prediction via quotient regression. An extensible bundle ecosystem provides pre-built model implementations that follow a consistent trainer--predictor pattern and integrate with the core framework without modification. MIPCandy is open-source under the Apache-2.0 license and requires Python~3.12 or later. Source code and documentation are available at https://github.com/ProjectNeura/MIPCandy.

Figures (5)

• Figure 1: Training progress plots automatically generated by MIPCandy during a U-Net training run on the PH2 dermoscopy dataset. The validation score is the negated combined loss (Section \ref{['sec:training']}); higher values indicate better performance.
• Figure 2: Worst-case prediction previews automatically saved during training. The framework identifies the validation case with the lowest score and generates overlays comparing the ground truth (b) and model prediction (c) against the input image (a). This example is from a U-Net trained on PH2.
• Figure 3: 3D volume previews rendered via PyVista. MIPCandy automatically generates 3D visualizations of labels and predictions for volumetric segmentation tasks.
• Figure 4: Notion frontend integration. MIPCandy automatically logs experiment metadata, progress, and scores to a Notion database.
• Figure 5: Console interface during training state recovery. The output shows a single epoch after recovery: sanity check, training metrics with a structured summary table, per-case validation metrics with per-class statistics, score prediction with ETC, and checkpoint management.