Spatial-temporal Hierarchical Reinforcement Learning for Interpretable Pathology Image Super-Resolution

TL;DR

STAR-RL reimagines pathology image super-resolution as a Markov decision process and solves it with a spatial-temporal hierarchical reinforcement learning framework. By introducing outer-loop patch selection via a spatial manager, inner-loop patch recovery via a patch worker, and a temporal manager to regulate termination, it achieves interpretable, resource-aware SR that preserves fine biological details. Quantitative and qualitative results on HistoSR demonstrate superior local-structure metrics (FSIM, GMSD) and competitive global metrics, with ablations validating each component. The method generalizes across degradations and sizes and shows promise for improving diagnostic performance in tasks like tumor recognition, including gigapixel whole-slide images.

Abstract

Pathology image are essential for accurately interpreting lesion cells in cytopathology screening, but acquiring high-resolution digital slides requires specialized equipment and long scanning times. Though super-resolution (SR) techniques can alleviate this problem, existing deep learning models recover pathology image in a black-box manner, which can lead to untruthful biological details and misdiagnosis. Additionally, current methods allocate the same computational resources to recover each pixel of pathology image, leading to the sub-optimal recovery issue due to the large variation of pathology image. In this paper, we propose the first hierarchical reinforcement learning framework named Spatial-Temporal hierARchical Reinforcement Learning (STAR-RL), mainly for addressing the aforementioned issues in pathology image super-resolution problem. We reformulate the SR problem as a Markov decision process of interpretable operations and adopt the hierarchical recovery mechanism in patch level, to avoid sub-optimal recovery. Specifically, the higher-level spatial manager is proposed to pick out the most corrupted patch for the lower-level patch worker. Moreover, the higher-level temporal manager is advanced to evaluate the selected patch and determine whether the optimization should be stopped earlier, thereby avoiding the over-processed problem. Under the guidance of spatial-temporal managers, the lower-level patch worker processes the selected patch with pixel-wise interpretable actions at each time step. Experimental results on medical images degraded by different kernels show the effectiveness of STAR-RL. Furthermore, STAR-RL validates the promotion in tumor diagnosis with a large margin and shows generalizability under various degradations. The source code is available at https://github.com/CUHK-AIM-Group/STAR-RL.

Figures (8)

• Figure 1: (a) The unexpected pattern generated by existing black-box method (PathSRGANma2020pathsrgan and SWD-Net chen2020joint); (b) SR performance gap with various metrics between black-box method and the proposed STAR-RL; (c) The cytoplasm and nucleus are representative cellular components. The cytoplasm with large-scale structure displays the smooth texture, which is easy to super-resolve. In contrast, the nucleus contains intricate biological details that are challenging to accurately restore due to significant information loss.
• Figure 2: The overview of Spatial-Temporal hierARchical Reinforcement Learning (STAR-RL) framework. It includes higher-level spatial Manager (spM), temporal Manager (tpM) and lower-level patch worker (PW). The black solid and yellow dashed lines represent the forward and reward propagation. The grey circle with C indicates the patch crop operation.
• Figure 3: The overview of unrolled Spatial-Temporal hierARchical Reinforcement Learning (STAR-RL) framework, consisting of an outer loop for patch selection and several inner loops for patch recovering.
• Figure 4: The architecture of (a) higher-level spatial manager and (b) lower-level patch worker.
• Figure 5: The statistical analysis between STAR-RL and existing SR methods for two evaluation metrics, i.e. (a) SSIM, (b) PSNR, (c) FSIM and (d) GMSD. A box plot is used to visualize the 5th $\sim$ 95th percentiles of results, while deviation outliers are represented as individual points to better observe the distribution.