WaveOrder: A differentiable wave-optical framework for scalable biological microscopy with diverse modalities

TL;DR

WaveOrder addresses the trade-offs in biological microscopy by providing a differentiable, physics-informed forward-model framework that unifies linear image formation across modalities and geometries. Through a computational-graph representation and physics-guided machine learning, it auto-tunes shift-variant transfer functions and solves blind deconvolution on tiled fields of view, enabling scalable, multi-contrast reconstructions from organelles to whole organisms. The approach yields both scalar (phase, density) and vector (polarization, orientation) reconstructions, improving segmentation, phenotyping, and cross-modal correlations in workflows spanning cells to zebrafish embryos. By delivering open-source, multi-interface tooling (Napari GUI, CLI, API, and browser demos) WaveOrder democratizes quantitative correlative imaging and sets the stage for future extensions to multi-scattering and real-time adaptive microscopy.

Abstract

Correlative computational microscopy can accelerate imaging and modeling of cellular dynamics by relaxing trade-offs inherent to dynamic imaging. Existing computational microscopy frameworks are either specialized or overly generic, limiting use to fixed configurations or domain experts. We introduce WaveOrder, a generalist wave-optical framework for imaging the architectural order of biomolecules. WaveOrder reconstructs diverse specimen properties from multi-channel acquisitions, with or without fluorescence. It provides a unified representation of linear optical properties and differentiable physics-based image formation models spanning widefield, confocal, light-sheet, and oblique label-free geometries. WaveOrder uses physics-informed ML to auto-tune model parameters and solve blind shift-variant restoration problems. This open-source, PyTorch-based framework enables scalable quantitative imaging across scales from organelles to adult zebrafish, and improves restoration of cellular structures in high-throughput experiments. We validate WaveOrder on diverse imaging applications, demonstrating its ability to recover biomolecular structure beyond the limits of existing approaches.

Figures (5)

• Figure 1: WaveOrder models and reconstructs multi-contrast volumetric microscopy data.(a) The framework models imaging with (i) microscopes equipped with (ii) an incoherent source with a programmable amplitude mask, tunable spectral/polarization filters, and a polarization-sensitive detector. Label-free contrast arises from (iii) direct-scatter interference, and fluorescence contrast from (iv) filtered scatter-scatter interference. Coherent scatterers are represented as (v) a volume of scattering potential tensors, and fluorescent emitters as (vi) a volume of dipole emission moments. (b) Parametrized transfer functions generalize across microscope geometries and contrast types. Transfer function colors indicate the relative phase of complex values, see inset color wheel. (c) WaveOrder consists of (i) representations of material properties $\mathbf{f}$ operated on by (ii) parametrized, physics-informed models $\mathcal{H}_{\boldsymbol{\theta}}$ to simulate (iii) multi-contrast multi-channel volumetric data $\mathbf{d}$. WaveOrder applies (iv) pseudo-inverse operators $\mathcal{H}_{\boldsymbol{\theta}}^{+}$ to (v) estimate specimen properties. Transfer function parameters $\boldsymbol{\theta}$ and the specimen properties $\mathbf{\hat{f}}$ are refined iteratively. (d) WaveOrder restores scalar datasets (e.g. (i) label-free at 5$\times$, (ii) 20$\times$, and (iii) confocal fluorescence) and (e) vectorial datasets (e.g. (i) polarization-resolved acquisitions use (ii) a bank of transfer functions to recover (iii) phase and birefringence). Scale bars: 25 µ m.
• Figure 2: Tiling and auto-tuning enable shift-variant blind deconvolutions. Large fields of view often exhibit shift-variant contrast because of variations in the transfer functions, making reconstruction a blind deconvolution problem. We address this by (a)(i) dividing the field of view into overlapping, approximately shift-invariant tiles. We reconstruct each tile using a physics-guided imaging model (ii) parameterized by a vector $\boldsymbol{\theta}$ describing optical misalignments and aberrations (illustrated for eight common misalignments). From these parameters we compute (iii) transfer functions $\mathcal{H}_{\boldsymbol{\theta}}$ (illustrated for phase-from-defocus with its illumination and detection pupils). We use the model as input to a (iv) Tikhonov-regularized least-squares problem to estimate $\mathbf{\hat{f}}$ in a single step. We compute a scalar loss on the reconstruction (illustrated for a mid-band spatial frequency loss) and update $\boldsymbol{\theta}$ by backpropagation. (b) We demonstrate on optical-pooled screen data from (i) a full 35 mm well. (ii--iii) Edge tiles show oblique-illumination contrast while central tiles show on-axis contrast. (iv) Nominal reconstructions fail on edge tiles, while (v) auto-tuned reconstructions recover consistent contrast and (vi) simultaneously estimate the underlying illumination tilt. (vii) CellPose segmentation performed best on the auto-tuned reconstruction. Scale bars: (b, i) 35 mm; (b, ii) 1 mm; (b, iv--v) 250 µ m. Pupil and transfer function colors indicate the relative size and phase of complex values, see color wheel in (a, iii).
• Figure 3: Flexible scalar reconstructions enable diverse biological applications.(a)(i) Thin label-free cells imaged in transmission suffer from low contrast, in-focus disappearance, and ambiguity in interpreting intensity values. Acquiring a through-focus stack together with (ii--iii) WaveOrder's auto-tuned reconstructions enables sharp single-plane reconstructions that summarize out-of-focus information, improve SNR, and remove the sign ambiguity of raw data. (iv) Comparing thresholded Frangi-filtered mitochondria segmentations from label-free and simultaneously acquired fluorescent mitochondrial imaging demonstrates the improved segmentation performance of a single-plane phase reconstruction. (b)(i) Zebrafish neuromasts imaged using (top) light-sheet illumination and oblique-detection configuration, typical of oblique-plane microscope geometries, yields contrast similar to a more typical (bottom) on-axis label-free and widefield fluorescence configuration. (ii--iii) WaveOrder enables reconstructions of all four contrast types, producing sharpened, higher-SNR images across modalities. (iv) Manual classification of mantle versus hair/support cells (top left) is similar to the fluorescence homogeneity % change after reconstruction (top right). Violin plots (bottom left) show improved separation between groups after reconstruction, with ROC curves (bottom right) confirming the improved classification. (c)(i) Cardiomyocytes imaged under nine oblique illumination geometries do not clearly reveal sarcomere architecture in the raw data alone. (ii--iii) WaveOrder's reconstruction routines enable visualization of z-discs from a single volume (blue outlines) and show further contrast enhancement when all nine illuminations are combined into a single phase estimate (red outlines). Transfer functions are in a single $9\times 1$ column vector, arranged into a $3\times 3$ grid. (iv) Sarcomere architecture (top, created with BioRender) and profiles (bottom) through the raw data (light blue), single-aperture reconstruction (dark blue), and multi-aperture reconstruction (red) demonstrate the improved contrast. Scale bars: (a--b) 10 µ m; (c) 5 µ m.
• Figure 4: Scalar reconstructions enable zebrafish phenotyping and characterization. A living $\sim$20 hpf zebrafish embryo imaged with simultaneous light-sheet fluorescence and label-free contrast. (a) Representative raw slices at different depths with ROIs indicated for panels (b--d). (b)(i) Tail, notochord, somites, and cellular structures visible in label-free and fluorescent channels; (ii) reconstruction improves contrast in both modalities; (iii) anatomical annotations enable tail straightening and quantification of Mezzo:GFP. (c)(i) Raw label-free data show ambiguous vacuole contrast; (ii) reconstruction resolves vacuoles as low-density compartments; (iii) single-profile measurements reveal a posterior-anterior increase in vacuole size. (d)(i--ii) Reconstruction highlights the retinal boundary and CMZ patterning in complementary channels; (iii) annotations support computational unwrapping to assess basal-apical density differences. Scale bars:(a) 100 µ m; (b) 50 µ m; (c) 25 µ m; (d) 50 µ m. Abbreviations:Ventral, Dorsal, Posterior, Anterior.
• Figure 5: Multi-contrast vectorial reconstructions improve visualization and interpretation across length scales in (a) A549 cells, (b) zebrafish stitched from seven fields of view (edges feathered to reduce background and tile edges), and (c--e) zebrafish regions of interest. (i) Orientation data (left) and reconstructions (right). (ii) Brightfield data (left) and phase reconstructions (right). (iii) Fluorescence data (left) and reconstructions (right). See also Videos 6 and 7. Scale bars: (a) 25 µ m; (b) 200 µ m; (c--e) 100 µ m.