From Performance to Practice: Knowledge-Distilled Segmentator for On-Premises Clinical Workflows

TL;DR

This work tackles the challenge of deploying high-accuracy medical image segmentation in on-premises clinical workflows with fixed hardware. It introduces a logit-based knowledge-distillation framework that transfers the performance of a large nnU-Net teacher to a scalable family of compact students that preserve the deployment interface. The distilled models maintain near-teacher accuracy under aggressive parameter reduction (up to 94%) while achieving substantial CPU latency gains (up to 67%), and they generalize to cross-modality abdominal CT (BTCV) without changes to the inference pipeline. The results demonstrate a practical, deployment-oriented path to translate research-grade segmentation into reliable, maintainable, on-premises components suitable for multi-service health systems, with qualitative and quantitative improvements in boundary stability and structural fidelity under constrained resources.

Abstract

Deploying medical image segmentation models in routine clinical workflows is often constrained by on-premises infrastructure, where computational resources are fixed and cloud-based inference may be restricted by governance and security policies. While high-capacity models achieve strong segmentation accuracy, their computational demands hinder practical deployment and long-term maintainability in hospital environments. We present a deployment-oriented framework that leverages knowledge distillation to translate a high-performing segmentation model into a scalable family of compact student models, without modifying the inference pipeline. The proposed approach preserves architectural compatibility with existing clinical systems while enabling systematic capacity reduction. The framework is evaluated on a multi-site brain MRI dataset comprising 1,104 3D volumes, with independent testing on 101 curated cases, and is further examined on abdominal CT to assess cross-modality generalizability. Under aggressive parameter reduction (94%), the distilled student model preserves nearly all of the teacher's segmentation accuracy (98.7%), while achieving substantial efficiency gains, including up to a 67% reduction in CPU inference latency without additional deployment overhead. These results demonstrate that knowledge distillation provides a practical and reliable pathway for converting research-grade segmentation models into maintainable, deployment-ready components for on-premises clinical workflows in real-world health systems.

Figures (3)

• Figure 1: Deployment pathway of the compact distilled segmentation model within an on-premises clinical environment. Imaging data are retrieved from PACS and processed on a local inference node running the distilled model. Segmentation outputs are returned in standard clinical formats (e.g., NIfTI or DICOM-SEG) for visualization and downstream quantitative analysis. This figure illustrates a representative integration scenario and does not correspond to a specific experimental pipeline.
• Figure 2: Overview of the proposed logit-based knowledge distillation framework for deployment-oriented model compression. A high-capacity nnU-Net teacher produces softened outputs to supervise a compact student via a KL-divergence loss, in addition to standard segmentation supervision. The teacher is used only during training, while the distilled student model is deployed for on-premises inference, enabling efficiency gains without altering the inference pipeline.
• Figure 3: Representative qualitative comparisons across coronal, sagittal, and axial views. Ground truth annotations (top), non-distilled student predictions (middle), and distilled student predictions (bottom) are shown. Dashed regions highlight areas where differences between methods are visually apparent. The distilled model exhibits improved local consistency and boundary preservation relative to the non-distilled baseline, consistent with the quantitative trends.