HAMscope: a snapshot Hyperspectral Autofluorescence Miniscope for real-time molecular imaging

TL;DR

HAMscope presents a compact snapshot hyperspectral autofluorescence miniscope that encodes spectral information in every frame via a thin polymer diffuser and reconstructs 30-channel hyperspectral stacks or direct biomolecule maps using a probabilistic multi-pass U-Net with transformer attention. The approach delivers real-time, uncertainty-aware spectral imaging with per-pixel confidence and generalizes to out-of-distribution tissues, demonstrated in plant systems where lignin, chlorophyll, and suberin maps are recovered. Key contributions include a scalable architecture, direct biomolecule mapping from encoded frames, and quantitative metrics showing high fidelity (~MAE ≈ 0.0048) and improved spatial resolution after deconvolution. The work enables field-deployable, label-free biochemical imaging across plant biology and beyond, with potential extensions to neuroscience, pathology, and environmental monitoring.

Abstract

We introduce HAMscope, a compact, snapshot hyperspectral autofluorescence miniscope that enables real-time, label-free molecular imaging in a wide range of biological systems. By integrating a thin polymer diffuser into a widefield miniscope, HAMscope spectrally encodes each frame and employs a probabilistic deep learning framework to reconstruct 30-channel hyperspectral stacks (452 to 703 nm) or directly infer molecular composition maps from single images. A scalable multi-pass U-Net architecture with transformer-based attention and per-pixel uncertainty estimation enables high spatio-spectral fidelity (mean absolute error ~ 0.0048) at video rates. While initially demonstrated in plant systems, including lignin, chlorophyll, and suberin imaging in intact poplar and cork tissues, the platform is readily adaptable to other applications such as neural activity mapping, metabolic profiling, and histopathology. We show that the system generalizes to out-of-distribution tissue types and supports direct molecular mapping without the need for spectral unmixing. HAMscope establishes a general framework for compact, uncertainty-aware spectral imaging that combines minimal optics with advanced deep learning, offering broad utility for real-time biochemical imaging across neuroscience, environmental monitoring, and biomedicine.

Figures (9)

• Figure 1: The HAMscope and reference imaging setup used for collecting training data. (A) The Miniscope V4 is modified by incorporating a thin polymer diffuser (not visible) at the image plane to enable spectral encoding (scale bar = 22 mm). A UV-LED excitation source, coupled with custom excitation and emission filters, supports label-free autofluorescence imaging of plant biomolecules such as lignin and chlorophyll. (B) A reference benchtop microscope configured in a 4F layout acquires ground-truth hyperspectral images using a tunable narrow-bandpass filter mounted on a motorized scanning stage. Reference and miniscope data are recorded simultaneously as a paired dataset. (C) The model takes one HAMscope image (scale bar = $200\mu$m) as input. A multi-U-Net generator produces a predicted hyperspectral stack or a compound-specific classification map derived from the spectral data. The generator also produces a per-pixel uncertainty map, enabling confidence estimation without ground-truth. Following the terminology in weigert2018, we refer to this as the hyperspectral scale. Paired input–output datasets (HAMscope and ground-truth hyperspectral images) are used to train the model. A Laplacian negative log-likelihood (NLL) loss guides the probabilistic predictions, while an adversarial discriminator further encourages perceptual realism in the output.
• Figure 2: Representative results from the HAMscope. (A) Reconstruction of the 572 nm spectral channel for a poplar stem cross-section: ground-truth (left), model prediction (center), and per-pixel uncertainty (right), expressed as the standard deviation from the probabilistic model ensemble. (B) Predicted and ground-truth spectral profiles at three locations (marked by colored “+” symbols), showing close agreement. Solid lines represent ground-truth, dashed lines indicate predictions; shaded regions denote $\pm2$ standard deviations. (C) Left: Full 30-channel hyperspectral image predicted from a single miniscope frame. Center: Predicted spectra at pixel (256, 256) from five independently trained probabilistic models compared with ground-truth. Right: Probability density functions (Laplacian) for the 572 nm channel, with the ground-truth value indicated. The ensemble prediction peaks closely align with the ground-truth, reflecting high model confidence. The results here used a single U-net model with instance normalization.
• Figure 3: Biomolecule mapping from predicted hyperspectral data across a tangential branch incision. (A) Autofluorescence spectra corresponding to lignin, chlorophyll, and other components, extracted from ground-truth hyperspectral images using the PoissonNMF plugin in ImageJ. These spectra were used uniformly across the training set to generate biomolecule maps from the predicted hyperspectral images. (B) Photograph of a tangential incision on a poplar branch, enabling analysis of biomolecular composition as a function of tissue depth. The dashed white box denotes the imaged region. (C) Composite image generated by stitching multiple fields of view, each processed using the PoissonNMF plugin in ImageJ applied to the predicted hyperspectral channels. The resulting color-coded map highlights the spatial distribution of lignin and chlorophyll across anatomical layers. Color legend indicates the spectral identity of each component.
• Figure 4: Representative results for direct biomolecule mapping from one miniscope image, without intermediate hyperspectral reconstruction. (A) Recorded HAMscope image. (B) Ground-truth biomolecule map derived from the corresponding ground-truth hyperspectral images. (C–F) Predicted biomolecule maps using deep learning models trained on paired HAMscope and ground-truth biomolecule-mapping data. Mean absolute error (MAE) is indicated for each model.
• Figure 5: Spatial resolution assessment of HAMscope reconstructions. (A–C) Ground-truth image at 526 nm (A), corresponding 2D Fourier transform (B), and azimuthally averaged power spectrum (C). (D–F) Predicted image from the hyperspectral U-Net (D), its Fourier transform (E), and power spectrum (F). (G–I) Same spectral slice after space-variant Wiener deconvolution Ingold2025 (G), with corresponding Fourier transform (H) and power spectrum (I). Spatial resolution was quantified by identifying the spatial frequency at which the log power spectrum dropped below a 0.1 threshold. Averaged over 100 test samples and 30 spectral channels, the network reconstructions achieved resolutions of 10.22 $\mu$m before and 6.74 $\mu$m after deconvolution, compared to 9.13 $\mu$m for the corresponding ground-truth image. Note that the ground-truth images were acquired using a different microscope configuration (objective and detector). Full resolution analysis is provided in Supplementary Fig. S10.