Deep learning based Image Compression for Microscopy Images: An Empirical Study

TL;DR

The paper tackles the challenge of massive microscopy datasets by evaluating both classic and deep-learning–based image compression methods through a two-phase pipeline that jointly optimizes Rate-Distortion and downstream label-free prediction performance. It leverages a hiPSC bright-field to fluorescence paradigm and employs multiple DL codecs from the CompressAI toolbox, comparing them against traditional codecs under $L = R + \lambda D$ with standard perceptual metrics. The study finds that AI-based compression generally outperforms classic methods in rate–distortion and preserves 2D label-free predictions with minimal degradation, though 3D tasks are more fragile and require compression-aware training. These results inform data infrastructure planning for bioimaging and highlight the importance of considering compression effects when deploying downstream AI analysis workflows.

Abstract

With the fast development of modern microscopes and bioimaging techniques, an unprecedentedly large amount of imaging data are being generated, stored, analyzed, and even shared through networks. The size of the data poses great challenges for current data infrastructure. One common way to reduce the data size is by image compression. This present study analyzes classic and deep learning based image compression methods, and their impact on deep learning based image processing models. Deep learning based label-free prediction models (i.e., predicting fluorescent images from bright field images) are used as an example application for comparison and analysis. Effective image compression methods could help reduce the data size significantly without losing necessary information, and therefore reduce the burden on data management infrastructure and permit fast transmission through the network for data sharing or cloud computing. To compress images in such a wanted way, multiple classical lossy image compression techniques are compared to several AI-based compression models provided by and trained with the CompressAI toolbox using python. These different compression techniques are compared in compression ratio, multiple image similarity measures and, most importantly, the prediction accuracy from label-free models on compressed images. We found that AI-based compression techniques largely outperform the classic ones and will minimally affect the downstream label-free task in 2D cases. In the end, we hope the present study could shed light on the potential of deep learning based image compression and the impact of image compression on downstream deep learning based image analysis models.

Figures (5)

• Figure 1: The workflow of a typical learned-based lossy image compression. The raw image $\mathbf{x}$ is fed into the encoder $f$ and obtain the low-dimensional latent representation $\mathbf{y}$. Then the lossless entropy coder can further exploit the information redundency: $\mathbf{y}$ will be firstly quantized to $\mathbf{z} \in \mathbb{Z}^n$, and then compressed to the bitstream $\mathbf{b}$ by the entropy encoder $f_e$. This bitstream can be stored for transmission or further decompression. The corresponding entropy decoder $g_e$ is responsible for the decompression and yield the reconstructed latent representation $\mathbf{\hat{y}}$. Lastly, $\mathbf{\hat{y}}$ is transmitted to the neural decoder $g$, yielding the reconstructed image $\mathbf{\hat{x}}$. The loss function of the system is composed of 2 parts: distortion $\mathcal{D}$ and rate $\mathcal{R}$. Distortion represents the reconstruction quality (e.g. SSIM between $\mathbf{x}$ and $\mathbf{\hat{x}}$) while rate focuses more on the compression ability. $\lambda$ acts as the hyper-parameter to balance the Rate-Distortion trade-off.
• Figure 2: Overview of our proposed evaluation pipeline. The objective is to fully estimate the compression performance of different compression algorithms (denoted as $g \circ f$) in the bioimage field and investigate their influence to the downstream AI-based bioimage analysis tasks (e.g. label-free task in this study, denoted as $f_l$). The solid line represents data flow while the dash line means evaluation. The brightfield raw image $x$ will be compressed and decompressed: $\hat{x} = (g \circ f)(x) = g(f(x))$. Then we feed the reconstructed $\hat{x}$ to the label-free model $f_l$ to get the estimated fluorescent image $\hat{y}$: $\hat{y} = f_l(\hat{x})$. Meanwhile, normal prediction $y$ is also made by $f_l$ from the raw image $x$: $y = f_l(x)$. Regarding the evaluation, ①\\② exhibits the Rate-Distortion ability of the compression algorithm, ③\\④\\⑤ represents their influence to the downstream task $f_l$. Specifically, ① measures the reconstruction ability of the compression method while ② records the bit-rate and can reflect the compression ratio ability. ③ and ④ represents the prediction accuracy of the $f_l$ model using the raw image $x$ and the reconstructed image $\hat{x}$ as input, respectively. ⑤ measures the similarity between these two predictions.
• Figure 3: Visualization of 2D brightfield image compression result (first row, model: bmshj2018-factorized (MS-SSIM)) + downstream label-free model prediction (second row). The upper right compression result is visually plausible compared to the input, and the compressed prediction (bottom left) using the label-free model is very close to the original prediction (bottom middle), which suggests the minimal influence of the selected deep-learning based compression to the downstream task.
• Figure 4: The prediction result of the downstream label-free models trained with lossy/losslessly compressed images, respectively. The input is the lossy compressed bright-field images. (a) Prediction from a label-free model trained with losslessly compressed images SollmannFOMPrediction, (b) Prediction from a label-free model trained with JPEG XR compressed images, (c) The ground truth. The label-free model trained on uncompressed data fails to produce accurate results when applied to lossy compressed images, as evidenced by the visible artifacts. This highlights the incompatibility between the model trained on original data and the application of lossy compression.
• Figure 5: Visualization of 3D compression result based on the bmshj2018-factorized model