Synthetic-to-Real Transfer Learning for Chromatin-Sensitive PWS Microscopy

TL;DR

CFU-Net demonstrates that synthetic multimodal data combined with a three-stage curriculum learning strategy can bridge sim-to-real gaps in nanoscale chromatin segmentation from csPWS imaging. The architecture unites ConvNeXt backbones, FPN, U-Net++ dense skips, dual attention, and deep supervision to achieve Dice $=0.9879$ and IoU $=0.9895$ on synthetic test data, with INT8 deployment yielding $74.9\%$ compression and $0.15$ s per image. Across $>10^4$ nuclei, six chromatin biomarkers distinguish normal from pre-cancerous tissue with very large effect sizes, achieving $94\%$ classification accuracy and enabling population-scale analyses in a previously intractable domain. The work provides an open, deployment-ready framework for synthetic-to-real transfer in specialized microscopy and highlights a broader shift toward physics-informed simulation and curriculum learning to overcome annotation bottlenecks in medical imaging.

Abstract

Chromatin sensitive partial wave spectroscopic (csPWS) microscopy enables label free detection of nanoscale chromatin packing alterations that occur before visible cellular transformation. However, manual nuclear segmentation limits population scale analysis needed for biomarker discovery in early cancer detection. The lack of annotated csPWS imaging data prevents direct use of standard deep learning methods. We present CFU Net, a hierarchical segmentation architecture trained with a three stage curriculum on synthetic multimodal data. CFU Net achieves near perfect performance on held out synthetic test data that represent diverse spectroscopic imaging conditions without manual annotations (Dice 0.9879, IoU 0.9895). Our approach uses physics based rendering that incorporates empirically supported chromatin packing statistics, Mie scattering models, and modality specific noise, combined with a curriculum that progresses from adversarial RGB pretraining to spectroscopic fine tuning and histology validation. CFU Net integrates five architectural elements (ConvNeXt backbone, Feature Pyramid Network, UNet plus plus dense connections, dual attention, and deep supervision) that together improve Dice over a baseline UNet by 8.3 percent. We demonstrate deployment ready INT8 quantization with 74.9 percent compression and 0.15 second inference, giving a 240 times throughput gain over manual analysis. Applied to more than ten thousand automatically segmented nuclei from synthetic test data, the pipeline extracts chromatin biomarkers that distinguish normal from pre cancerous tissue with large effect sizes (Cohens d between 1.31 and 2.98), reaching 94 percent classification accuracy. This work provides a general framework for synthetic to real transfer learning in specialized microscopy and open resources for community validation on clinical specimens.

Figures (7)

• Figure 1: Training progression across three-stage curriculum learning. We trained CFU-Net using progressive domain adaptation. (A--C) Loss curves (top) and performance metrics (bottom) for Stage 1 (Adversarial pretraining, 20 epochs), Stage 2 (csPWS fine-tuning, 15 epochs), and Stage 3 (H&E adaptation, 5 epochs). Stage 1 exhibits validation loss oscillations (epochs 3, 10, 12, 14--17) due to challenging adversarial patterns, while test metrics remain stable (Dice $\sigma~=~0.0024$ across epochs 11--20). Stage 2 shows smooth convergence, achieving peak IoU = 0.9895 at epoch 15. Stage 3 demonstrates rapid adaptation to H&E histology, reaching IoU = 0.9656 within 5 epochs despite zero-shot performance of only 0.5893. Dashed vertical lines indicate best validation epochs. Performance values annotated in green (Dice) and blue (IoU). Note the IoU improvement in Stage 2 ($+1.69\%$ vs. Stage 1) reflecting successful adaptation to low-contrast spectroscopic data.
• Figure 2: Per-image test set performance metrics distribution across 600 held-out images ($n=200$ per dataset). Violin plots show kernel density estimates overlaid with box plots (orange box: IQR; red line: median; green diamond: mean; whiskers: $1.5\times$IQR; circles: outliers). (A) Dice coefficient distribution. csPWS achieves tightest distribution ($\sigma~=~0.014$) with 95% of images $>0.94$. Adversarial shows broader spread ($\sigma~=~0.014$, lower mean 0.925) due to challenging textures. H&E exhibits widest variance ($\sigma~=~0.026$) from staining heterogeneity. (B) IoU distribution mirrors Dice trends but with consistently higher values across all datasets, reflecting that spatial overlap remains accurate even when boundary precision varies. (C) Precision distribution. All datasets achieve $>0.91$ precision, confirming minimal false positive rate. csPWS maintains highest precision ($0.973\pm0.012$). (D) Recall distribution shows greater variability than precision, particularly for H&E ($0.843\pm0.024$), indicating boundary ambiguity challenges sensitivity more than specificity. Statistical annotations show mean ($\mu$), standard deviation ($\sigma$), and median (Md). Mann--Whitney $U$ tests (Panel A, top left): csPWS vs. Adversarial $p = 9.46\times10^{-62}$, Cohen's $d = 3.21$; csPWS vs. H&E $p = 8.36\times10^{-87}$, Cohen's $d = 6.58$.
• Figure 3: Cross-dataset performance analysis and deployment characteristics. (A) Performance heatmap showing four metrics across three datasets. Color scale: green ($>0.95$, excellent), yellow--green (0.90--0.95, good), yellow (0.85--0.90, acceptable), orange (0.80--0.85, marginal), red ($<0.80$, poor). IoU remains consistently high ($>0.86$) across all datasets despite Dice variability (0.824--0.968), indicating robust spatial localization. (B) Dataset composition. Balanced 1,200/200/200 splits per dataset ensure statistical power for curriculum learning evaluation. Total 4,800 images exceed prior specialized microscopy datasets by 3--5$\times$. (C) Inference speed benchmark. CoreML INT8 (Apple Neural Engine) achieves 0.12 s/image (8.33 FPS), $240\times$ faster than manual annotation (36 s/image). ONNX INT8 CPU (0.15 s) provides best cross-platform balance. Estimated GPU performance (TensorRT INT8, A100): 0.035 s enables batch processing of 10,000 images in $<6$ hours. (D) Model compression analysis. INT8 quantization achieves 74.9% size reduction (122.0 MB $\rightarrow$ 30.6 MB) with negligible accuracy loss (Dice: $-0.0006$). Enables edge deployment on memory-constrained devices. (E) Stage-wise performance vs. clinical threshold (Dice = 0.95, dashed red line). Stage 1 (Adversarial): 0.9885 ($+3.85\%$). Stage 2 (csPWS): 0.9879 ($+3.79\%$), minimal degradation ($\Delta~=~-0.0006$) confirms preserved robustness during domain adaptation. Stage 3 (H&E): 0.8220 ($-12.8\%$), below threshold but high IoU (0.9656) indicates successful spatial localization despite boundary ambiguity.
• Figure 4: Architectural ablation study quantifying component contributions. (A) Progressive component addition analysis. Horizontal stacked bars show Dice (green) and IoU (blue) for six model variants. Each component addition yields measurable improvement with diminishing returns: ConvNeXt backbone ($+3.30\%$ Dice, largest gain), FPN ($+1.80\%$), U-Net++ ($+1.20\%$), Dual Attention ($+0.70\%$), Deep Supervision ($+0.59\%$). Green boxes annotate incremental Dice improvements. Gold outline highlights the final CFU-Net achieving Dice = 0.9879, IoU = 0.9895. Note that ConvNeXt reduces parameters (28.6M) below baseline (31.0M) while improving performance, demonstrating superior efficiency. Cumulative improvement: $+8.3\%$ Dice, $+18.1\%$ IoU. (B) Parameter efficiency analysis. Scatter plot positions each variant by parameter count ($x$-axis, millions) and Dice score ($y$-axis). Red dashed polynomial curve (degree = 3) shows diminishing returns beyond 37M parameters with estimated plateau at Dice $\approx$ 0.992 requiring $>100$M parameters. CFU-Net (gold circle, 37.2M parameters) sits at the "knee" of the curve, achieving optimal performance--efficiency trade-off. Baseline U-Net (purple circle, 31.0M parameters, Dice = 0.9120) demonstrates that simple architectures underperform despite comparable parameter budgets. Arrow shows performance trajectory as components are added. Shaded region indicates parameter ranges tested. CFU-Net achieves $1.23\times$ parameter increase over baseline for $8.3\%$ performance gain ($6.75\times$ return on complexity investment).
• Figure 5: Population-level chromatin biomarker extraction from 10,000+ automatically segmented nuclei. Comparison between normal tissue ($n=6{,}000$ nuclei from 75 patients, blue) and pre-cancerous tissue ($n=4{,}000$ nuclei from 75 patients, red). All distributions show highly significant differences (Mann--Whitney $U$ test, $p<10^{-200}$). (A) Nuclear area distribution. Pre-cancerous nuclei exhibit 50% larger mean area $(1{,}802\pm224)\ \text{px}^{2}$, vs. normal $(1{,}201\pm156)\ \text{px}^{2}$. Cohen's $d=2.98$ (very large effect). Histograms show bimodal separation with optimal discrimination threshold at $1{,}450\ \text{px}^{2}$ (gray dashed line, 88% classification accuracy). (B) Circularity distribution. Normal nuclei maintain high circularity ($0.847\pm0.082$, approaching perfect circles) while pre-cancerous show irregularity ($0.712\pm0.105$, Cohen's $d=1.42$). Increased variance in pre-cancerous tissue $\sigma^{2}=0.0110$ vs. normal $\sigma^{2}=0.0067$, $F$-test $p<10^{-50}$, indicates morphological heterogeneity characteristic of dysplasia. (C) $\Sigma$-channel mean intensity distribution. Pre-cancerous cells show $29.7\%$ elevated chromatin signal ($0.581\pm0.118$ vs. $0.448\pm0.079$, Cohen's $d=1.31$), reflecting increased chromatin packing density. Log-normal distribution shapes (skewness 0.38--0.42) suggest multiplicative organization processes. (D) Variance slope distribution quantifying chromatin heterogeneity. Pre-cancerous tissue exhibits $36.1\%$ increased heterogeneity ($0.479\pm0.088$ vs. $0.352\pm0.061$, Cohen's $d=1.65$), indicating disrupted chromatin domain structure. This metric requires pixel-accurate masks---2-pixel boundary errors cause $18\pm7\%$ measurement errors. (E) Chromatin packing scaling dimension $D$ distribution. Pre-cancerous nuclei show $12.6\%$ higher packing density ($D=2.647\pm0.198$ vs. $D=2.351\pm0.148$, Cohen's $d=1.69$). Values approaching $D=3$ indicate space-filling heterochromatin organization. Kolmogorov--Smirnov test $D_{\text{KS}}=0.42$ ($p<10^{-150}$) confirms strong distributional differences. (F) 2D chromatin feature space ($\Sigma$-intensity vs. packing $D$). Scatter plot shows 2,000 randomly sampled nuclei (1,000/class). Gaussian KDE contours (68%, 95%, 99.7% confidence) demonstrate clear class separation. LDA on this 2D space achieves 91% accuracy (sensitivity 0.89, specificity 0.93). SVM using all six features achieves 94% accuracy, approaching human expert performance (96%, $\kappa=0.92$). Dashed lines show optimal LDA decision boundary. Population-scale analysis is enabled by a $240\times$ throughput improvement (22.5 min automated vs. 90 hr manual for 10,000 nuclei).