Comparative Evaluation of Deep Learning-Based and WHO-Informed Approaches for Sperm Morphology Assessment

TL;DR

The paper tackles the subjectivity and variability of sperm morphology assessment by comparing a deep learning image-based approach (HuSHeM CNN) against a WHO+SIRI clinical baseline. HuSHeM demonstrates superior discrimination, calibration, and net clinical benefit on an independent test cohort, achieving a ROC-AUC of $0.975$ and PR-AUC of $0.993$, vastly outperforming the baseline (ROC-AUC $=0.721$, PR-AUC $=0.097$). The study also provides qualitative evidence via Grad-CAM that the CNN focuses on meaningful head morphology features, and it emphasizes the decision-support nature of AI in fertility screening. These findings suggest that image-based AI can offer objective, reproducible, and clinically valuable sperm morphology assessments, though prospective multicenter validation is needed before routine clinical deployment.

Abstract

Assessment of sperm morphological quality remains a critical yet subjective component of male fertility evaluation, often limited by inter-observer variability and resource constraints. This study presents a comparative biomedical artificial intelligence framework evaluating an image-based deep learning model (HuSHeM) alongside a clinically grounded baseline derived from World Health Organization criteria augmented with the Systemic Inflammation Response Index (WHO(+SIRI)). The HuSHeM model was trained on high-resolution sperm morphology images and evaluated using an independent clinical cohort. Model performance was assessed using discrimination, calibration, and clinical utility analyses. The HuSHeM model demonstrated higher discriminative performance, as reflected by an increased area under the receiver operating characteristic curve with relatively narrow confidence intervals compared to WHO(+SIRI). Precision-recall analysis further indicated improved performance under class imbalance, with higher precision-recall area values across evaluated thresholds. Calibration analysis indicated closer agreement between predicted probabilities and observed outcomes for HuSHeM, while decision curve analysis suggested greater net clinical benefit across clinically relevant threshold probabilities. These findings suggest that image-based deep learning may offer improved predictive reliability and clinical utility compared with traditional rule-based and inflammation-augmented criteria. The proposed framework supports objective and reproducible assessment of sperm morphology and may serve as a decision-support tool within fertility screening and referral workflows. The proposed models are intended as decision-support or referral tools and are not designed to replace clinical judgment or laboratory assessment.

Figures (5)

• Figure 1: Overview of the study pipeline comparing image-based deep learning sperm morphology assessment (HuSHeM CNN) with the guideline-based clinical baseline incorporating World Health Organization (WHO) morphology criteria and the Systemic Inflammation Response Index (SIRI). The pipeline illustrates data acquisition, model construction, evaluation methodology, and clinical utility analysis. The HuSHeM convolutional neural network operates directly on sperm head images to learn discriminative morphological features, whereas the WHO(+SIRI) model relies on handcrafted clinical predictors derived from manual morphology assessment and systemic inflammatory markers. Model performance is evaluated using complementary metrics including discrimination (ROC-AUC, PR-AUC), calibration (agreement between predicted probabilities and observed outcomes), and clinical utility (decision curve analysis).
• Figure 2: Training and validation loss curves for the HuSHeM convolutional neural network. The progressive reduction and convergence of cross-entropy loss across epochs indicate stable optimization and no evidence of severe overfitting, supporting reliable feature learning from sperm morphology images.
• Figure 3: Calibration curves comparing predicted probabilities with observed outcome frequencies for the HuSHeM convolutional neural network and the WHO(+SIRI) clinical model. The proximity of the HuSHeM curve to the ideal diagonal indicates improved agreement between predicted probabilities and observed outcomes at clinically relevant extremes, supporting more reliable risk estimation compared with guideline-based clinical features.
• Figure 4: Decision curve analysis illustrating net clinical benefit across a range of threshold probabilities for the HuSHeM CNN and the WHO(+SIRI) clinical model. The HuSHeM model demonstrates consistently higher net benefit relative to treat-all and treat-none strategies, indicating greater potential value for clinical decision support.
• Figure 5: Receiver operating characteristic (ROC) and precision–recall (PR) curves comparing diagnostic discrimination performance of the HuSHeM CNN and the WHO(+SIRI) clinical model. ROC curves summarize threshold-independent discrimination, while PR curves emphasize performance under class imbalance. HuSHeM consistently demonstrates superior discriminative capacity across both evaluation paradigms.