MONAI Label: A framework for AI-assisted Interactive Labeling of 3D Medical Images

TL;DR

The paper addresses the bottleneck of limited annotated 3D medical datasets by introducing MONAI Label, a free, open-source framework for AI-assisted interactive labeling. It combines interactive (DeepGrow, DeepEdit, Scribbles-based) and non-interactive automated segmentation approaches, deployable as RESTful apps via MONAI Label Server, and supports 3D Slicer and OHIF frontends with DICOMWeb integration. A key contribution is the inclusion of active learning with dropout-based uncertainty estimation to efficiently select unlabeled data for annotation, plus a heuristic planner for hardware-aware training/inference. The framework is demonstrated through experiments (e.g., spleen and left atrium segmentation) showing substantial reductions in annotation time and enhanced clinician collaboration, and case studies (Telestroke, Neurosurgical Atlas) illustrating real-world impact. Overall, MONAI Label facilitates rapid, scalable creation and deployment of AI-augmented labeling tools with extensible interfaces and workflows.

Abstract

The lack of annotated datasets is a major bottleneck for training new task-specific supervised machine learning models, considering that manual annotation is extremely expensive and time-consuming. To address this problem, we present MONAI Label, a free and open-source framework that facilitates the development of applications based on artificial intelligence (AI) models that aim at reducing the time required to annotate radiology datasets. Through MONAI Label, researchers can develop AI annotation applications focusing on their domain of expertise. It allows researchers to readily deploy their apps as services, which can be made available to clinicians via their preferred user interface. Currently, MONAI Label readily supports locally installed (3D Slicer) and web-based (OHIF) frontends and offers two active learning strategies to facilitate and speed up the training of segmentation algorithms. MONAI Label allows researchers to make incremental improvements to their AI-based annotation application by making them available to other researchers and clinicians alike. Additionally, MONAI Label provides sample AI-based interactive and non-interactive labeling applications, that can be used directly off the shelf, as plug-and-play to any given dataset. Significant reduced annotation times using the interactive model can be observed on two public datasets.

Figures (9)

• Figure 1: Overview of the MONAI Label platform: MONAI Label consists of three main high-level modules, namely (i) client, (ii) server, and (iii) Database/Datastores. On the client side, MONAI Label supports different graphical user interfaces (GUIs) (3D Slicer/OHIF) for data viewing and annotation. On the server side, AI-assisted interactive and non-interactive annotation methods
• Figure 2: Schema of the DeepGrow approach: The input tensor consists of the image and two tensors representing positive (Pos) and negative (Neg) clicks provided by the user.
• Figure 3: General schema of the DeepEdit approach: DeepEdit training process consists of a combination of two modes: the automatic segmentation mode and DeepGrow mode. For inference, the input tensor could be either the image with two zero-tensor (automatic segmentation mode) or an image with two tensors representing positive and negative clicks provided by the user (interactive mode or DeepGrow mode).
• Figure 4: Scribbles-based interactive segmentation in MONAI Label: Scribbles-based methods consist of two stages, namely (i) likelihood inference stage and, (ii) segmentation refinement stage. The likelihood can come from either an online model built using the image volume and user scribbles, or using a pre-trained deep learning model on image volume alone. The interactions are provided as foreground (FG) or background (BG) scribbles. Image volume, likelihood, and scribbles are then used in a refinement stage to refine the initial segmentation using an energy optimization approach, e.g. using GraphCut boykov2004experimental.
• Figure 5: Active learning cycle: An expert labeller annotates a volume and adds it to the training pool. A machine learning model is trained with the new annotation obtaining a model. This model allows for uncertainty estimation which is utilized to rank the unlabeled volumes in order to select a set of the most uncertain volumes. The selected uncertain data are labeled by the expert which are labeled and added to the training pool and the cycle can be repeated as many times till a model of desired performance is achieved.