A Contrastive Pre-trained Foundation Model for Deciphering Imaging Noisomics across Modalities

TL;DR

This work tackles the challenge of characterizing complex, device-dependent imaging noise by introducing Noisomics and the Contrastive Pre-trained (CoP) Foundation Model. By coupling a synthetic noise genome with contrastive pre-training, CoP learns a disentangled, content-agnostic noise representation that enables precise multi-parameter noise estimation with only $10^2$ samples, outperforming traditional supervised approaches by orders of magnitude. The approach demonstrates robust zero-shot generalization across 12 out-of-domain modalities and provides interpretable insights into how acquisition parameters influence noise via SHAP and Sankey visualizations, with practical demonstrations in consumer photography and depth-optimized in-vivo deep-tissue microscopy using depth-gained excitation. Overall, Noisomics reframes stochastic degradation as a diagnostic and optimization resource, offering a scalable framework for device-agnostic imaging diagnostics and protocol design with potential real-time applications and open-source tooling.

Abstract

Characterizing imaging noise is notoriously data-intensive and device-dependent, as modern sensors entangle physical signals with complex algorithmic artifacts. Current paradigms struggle to disentangle these factors without massive supervised datasets, often reducing noise to mere interference rather than an information resource. Here, we introduce "Noisomics", a framework shifting the focus from suppression to systematic noise decoding via the Contrastive Pre-trained (CoP) Foundation Model. By leveraging the manifold hypothesis and synthetic noise genome, CoP employs contrastive learning to disentangle semantic signals from stochastic perturbations. Crucially, CoP breaks traditional deep learning scaling laws, achieving superior performance with only 100 training samples, outperforming supervised baselines trained on 100,000 samples, thereby reducing data and computational dependency by three orders of magnitude. Extensive benchmarking across 12 diverse out-of-domain datasets confirms its robust zero-shot generalization, demonstrating a 63.8% reduction in estimation error and an 85.1% improvement in the coefficient of determination compared to the conventional training strategy. We demonstrate CoP's utility across scales: from deciphering non-linear hardware-noise interplay in consumer photography to optimizing photon-efficient protocols for deep-tissue microscopy. By decoding noise as a multi-parametric footprint, our work redefines stochastic degradation as a vital information resource, empowering precise imaging diagnostics without prior device calibration.

Figures (6)

• Figure 1: The Contrastive Pre-trained (CoP) framework for Noisomics.
• Figure 2: The Contrastive Pre-trained (CoP) framework for Noisomics.
• Figure 3: Quality evaluation of the relationships between image acquisition parameters and noise components in smartphone photography.
• Figure 4: Quantitative analysis of noise reduction by depth-gained excitation in deep-tissue three-photon microscopy using the CoP framework.
• Figure 5: Comparisons of the threshold accuracy of multi-noise categories classification in cross modalities OOD data.