Coordinate-conditioned Deconvolution for Scalable Spatially Varying High-Throughput Imaging

TL;DR

SV-CoDe addresses spatially varying blur in high-throughput, multi-view fluorescence imaging by replacing global SV operators with coordinate-conditioned convolutions that adapt locally to field-dependent aberrations. The two-stage Demixing-Net and Recon-Net leverage CoDe blocks and positional encodings to implement SV demixing and deconvolution with minimal memory, enabling patch-based training that decouples parameter count from FOV size. Trained solely on physics-based simulations, SV-CoDe demonstrates state-of-the-art reconstruction quality, uniform resolution across a 6.5 mm FOV, and robust generalization to bead phantoms, weakly scattering brain tissue, and freely moving C. elegans, while requiring roughly 10× less model size and 10× less training data than baselines. The approach offers a scalable, physics-aware solution for correcting SV blur in compact optical systems and holds potential for extension to 3D, volumetric imaging and other computational imaging applications.

Abstract

Wide-field fluorescence microscopy with compact optics often suffers from spatially varying blur due to field-dependent aberrations, vignetting, and sensor truncation, while finite sensor sampling imposes an inherent trade-off between field of view (FOV) and resolution. Computational Miniaturized Mesoscope (CM2) alleviate the sampling limit by multiplexing multiple sub-views onto a single sensor, but introduce view crosstalk and a highly ill-conditioned inverse problem compounded by spatially variant point spread functions (PSFs). Prior learning-based spatially varying (SV) reconstruction methods typically rely on global SV operators with fixed input sizes, resulting in memory and training costs that scale poorly with image dimensions. We propose SV-CoDe (Spatially Varying Coordinate-conditioned Deconvolution), a scalable deep learning framework that achieves uniform, high-resolution reconstruction across a 6.5 mm FOV. Unlike conventional methods, SV-CoDe employs coordinate-conditioned convolutions to locally adapt reconstruction kernels; this enables patch-based training that decouples parameter count from FOV size. SV-CoDe achieves the best image quality in both simulated and experimental measurements while requiring 10x less model size and 10x less training data than prior baselines. Trained purely on physics-based simulations, the network robustly generalizes to bead phantoms, weakly scattering brain slices, and freely moving C. elegans. SV-CoDe offers a scalable, physics-aware solution for correcting SV blur in compact optical systems and is readily extendable to a broad range of biomedical imaging applications.

Figures (8)

• Figure 1: System overview. (a) A multi-aperture miniscope uses an MLA as the sole imaging element; each microlens captures a distinct sub-FOV, and the sensor records an extended FOV with inter-view crosstalk. (b) Low-rank, SV forward model used for physics-based data synthesis. The simulated measurement is partitioned into multiplexed sub-views that serve as network inputs. (c) SV-CoDe pipeline: multiplexed sub-views and normalized spatial coordinates are cropped into patches and processed by a view-dependent Demixing-Net with CoDe blocks, followed by a Recon-Net that performs SV deconvolution to fuse views into a wide-FOV image. (d) SV-CoDe augments NAFNet with coordinate-conditioned spatial attention via a lightweight MLP, enabling SV reconstruction with minimal memory overhead and achieving higher PSNR with substantially fewer parameters than baseline methods.
• Figure 2: Network design. (a) Demixing module: The raw measurement is partitioned into microlens sub-views using chief-ray coordinates. Each sub-view is processed by a view-dependent CoDe block, and the resulting features are passed to a U-Net composed of shared CoDe blocks. (b) Reconstruction module: The demixed sub-views are fed into the reconstruction network, which begins with a CoDe block to perform lens-dependent SV deconvolution. Channel attention then fuses information across views, followed by a second U-Net stack to produce a wide-FOV, high-resolution reconstruction. (c) CoDe block: The CoDe block replaces SI convolution with a coordinate-conditioned operation. As shown in the zoom-in, pixel coordinates $(i,j)$ are positional encoded and mapped by a lightweight MLP to a per-channel SV mask that modulates the post-convolution features.
• Figure 3: Benchmarking results. (a) Simulation (cells): SV-CoDe achieves the highest PSNR/SSIM and produces visibly sharper, cleaner reconstructions than MultiWienerNet and SV-FourierNet. Insets show enlarged regions (white dashed boxes). (b) Experiment (resolution target): SV-CoDe, SV-FourierNet, and LSV-ADMM resolve the finest elements (yellow boxes), with SV-CoDe exhibiting higher contrast and fewer artifacts than the other methods. Annotated times report per-frame inference latency on a single GPU.
• Figure 4: Ablation on coordinate gate (CG) and positional encoding (PE). (a) Central-view demixing. Columns show the ground truth, w/o CG, w/o PE, and the proposed SV-CoDe. Removing CG leaves residual background, particularly near the field edge where spatial variance is strongest due to sensor truncation (red arrows). Both w/o PE and the full SV-CoDe yield comparable demixing quality. (b) Final reconstruction. While central-view demixing alone suffers from severe peripheral aberrations and vignetting, the reconstruction stage recovers fine details and effectively extends the usable FOV. Models w/o CG propagate demixing artifacts into the reconstruction (red arrows), whereas removing PE leads to visibly reduced resolution due to spectral bias (yellow arrows). By leveraging CG for SV deconvolution, SV-CoDe suppresses peripheral ghosting and achieves the highest visual fidelity, closely matching the ground truth.
• Figure 5: Resolution chracterization. Reconstructions of a fluorescent resolution target placed at multiple locations within the FOV (insets: yellow square marks the target position). Top row: reconstructions at different field positions. Bottom row: zooms of the white dashed regions; the yellow box marks the finest resolvable line pair. SV-CoDe maintains uniform resolution across different FOV positions.