Novel computational workflows for natural and biomedical image processing based on hypercomplex algebras

TL;DR

By leveraging quaternions and the two-dimensional orthogonal planes split framework, image processing workflows for natural and biomedical images, including natural and biomedical image recolorization, natural image decolorization, natural and biomedical image contrast enhancement, and computational restaining and stain separation in histological images are demonstrated.

Abstract

Hypercomplex image processing extends conventional techniques in a unified paradigm encompassing algebraic and geometric principles. This work leverages quaternions and the two-dimensional orthogonal planes split framework (splitting of a quaternion - representing a pixel - into pairs of orthogonal 2D planes) for natural/biomedical image analysis through the following computational workflows and outcomes: natural/biomedical image re-colorization, natural image de-colorization, natural/biomedical image contrast enhancement, computational re-staining and stain separation in histological images, and performance gains in machine/deep learning pipelines for histological images. The workflows are analyzed separately for natural and biomedical images to showcase the effectiveness of the proposed approaches. The proposed workflows can regulate color appearance (e.g. with alternative renditions and grayscale conversion) and image contrast, be part of automated image processing pipelines (e.g. isolating stain components, boosting learning models), and assist in digital pathology applications (e.g. enhancing biomarker visibility, enabling colorblind-friendly renditions). Employing only basic arithmetic and matrix operations, this work offers a computationally accessible methodology - in the hypercomplex domain - that showcases versatility and consistency across image processing tasks and a range of computer vision and biomedical applications. The proposed non-data-driven methods achieve comparable or better results (particularly in cases involving well-known methods) to those reported in the literature, showcasing the potential of robust theoretical frameworks with practical effectiveness. Results, methods, and limitations are detailed alongside discussion of promising extensions, emphasizing the potential of feature-rich mathematical/computational frameworks for natural and biomedical images.

Figures (19)

• Figure 1: Re-colorization in synthetic and natural images (24-bit). Synthetic image (a) containing red, green, and blue color pixel blocks (block size: $32^2$ pixels) with sample renditions: (a1) $q_-$, $\boldsymbol{f} = \boldsymbol{\mu_{1}}$, (a2) $q_-$, $\boldsymbol{f} = \boldsymbol{\mu_{2}}$, and (a3) $q_-$, $\boldsymbol{f} = \boldsymbol{\mu_{3}}$. Natural images (b-c) from the McGill color image database Olmos2004 with sample renditions: (b1) $q_+$, $\boldsymbol{f} = \boldsymbol{\mu_{2}}$, $\boldsymbol{g} = \boldsymbol{\mu_{11}}$, (b2) $q_-$, $\boldsymbol{f} = \boldsymbol{\mu_{3}}$, $\boldsymbol{g} = \boldsymbol{\mu_{8}}$, (b3) $q_-$, $\boldsymbol{f} = \boldsymbol{\mu_{2}}$, $\boldsymbol{g} = \boldsymbol{\mu_{3}}$, (c1) $q_-$, $\boldsymbol{f} = \boldsymbol{\mu_{1}}$, $\boldsymbol{g} = \boldsymbol{\mu_{2}}$, (c2) $q_-$, $\boldsymbol{f} = \boldsymbol{\mu_{1}}$, $\boldsymbol{g} = \boldsymbol{\mu_{10}}$, and (c3) $q_+$, $\boldsymbol{f} = \boldsymbol{\mu_{3}}$, $\boldsymbol{g} = \boldsymbol{\mu_{11}}$. Painting (d) with sample renditions: (d1) $q_-$, $\boldsymbol{f} = \boldsymbol{\mu_{2}}$, $\boldsymbol{g} = \boldsymbol{\mu_{5}}$, (d2) $q_-$, $\boldsymbol{f} = \boldsymbol{\mu_{5}}$, $\boldsymbol{g} = \boldsymbol{\mu_{8}}$, and (d3) $q_+$, $\boldsymbol{f} = \boldsymbol{\mu_{8}}$, $\boldsymbol{g} = \boldsymbol{\mu_{13}}$. Corresponding re-colorization images with user-defined values ($q_+$, sampling shown by arrows and expressed as pure unit quaternions) are shown in (a4-d4). Arrows correspond to pixel coordinates in (a-d); a single arrow corresponds to $\boldsymbol{f}$ while two arrows to $\boldsymbol{f}$ and $\boldsymbol{g}$. Pure unit quaternions $\boldsymbol{\mu_{1}}$-$\boldsymbol{\mu_{13}}$ are shown in Fig. \ref{['fig:m1m13']}. Images have $256^2$ pixels.
• Figure 2: De-colorization in synthetic and natural images (24-bit): (a) synthetic image comprising randomly colored pixel blocks (block size: $32^2$ pixels) and its corresponding 8-bit renditions; a1: $q_-$, $\boldsymbol{f} = \boldsymbol{\mu_{7}}$ and a2: $q_-$, $\boldsymbol{f}$, sampling shown by arrow in (a) and expressed as a pure unit quaternion. Likewise, images (b1-d1): $q_-$, $\boldsymbol{f} = \boldsymbol{\mu_{7}}$ and (b2-d2): $q_-$, $\boldsymbol{f}$ (sampling shown by arrows and expressed as pure unit quaternions) are the 8-bit renditions of Fig. \ref{['fig:img_recol']}(b-d), respectively. The arrows in (b1-d1) correspond to pixel coordinates in the color images of Fig. \ref{['fig:img_recol']}(b-d) for obtaining (b2-d2). Pure unit quaternions $\boldsymbol{\mu_{1}}$-$\boldsymbol{\mu_{13}}$ are shown in Fig. \ref{['fig:m1m13']}. Images have $256^2$ pixels.
• Figure 3: De-colorization in natural images (24-bit). The original images shown (a-c) are selected examples from the COLOR250 dataset Lu2014. The resulting images (8-bit) in a(1-4), b(1-4), and c(1-4) derive from methods L Hunt2011, O ITU-R_BT.2100, and proposed methods P1 and P2b (Table \ref{['tab:decol_comp']}), respectively. Displayed images are cropped and scaled versions ($250^2$ pixels) for presentation purposes.
• Figure 4: Contrast enhancement in natural and medical (CT) images (24-bit, 8-bit). The original images (a-d) are examples from the CEED2016 Qureshi2016Qureshi2017 and COVID-LDCT Afshar2021 datasets. The images in (a1-d1) derive from methods A-C Beghdadi1989Zuiderveld1994Mukherjee2008 (Table \ref{['tab:contr_comp']}). The images in (a2-d2) derive from the proposed method P (Table \ref{['tab:contr_comp']}). Histological close-up image sections (e-h) show an H&E stain, and single immunostains that exhibit approx. brown, green, and red colors, respectively. For (f-h), the counterstain exhibits approx. blue color. The images (e1-h1) derive from the proposed method P (Table \ref{['tab:contr_comp']}). Natural and medical (CT) images have $512^2$ pixels, and histology images have $250^2$ pixels.
• Figure 5: Re-colorization in histological images and their synthetic models (24-bit). Histological close-up image sections (a-e) show an H&E stain, single immunostains that exhibit approx. brown, green, and red colors, and a double immunostain exhibiting approx. brown and red colors, respectively. For (b-e), the counterstain exhibits approx. blue color. The sample renditions are: (a1) $q_+$, $\boldsymbol{f} = \boldsymbol{\mu_{5}}$, (a2) $q_+$, $\boldsymbol{f} = \boldsymbol{\mu_{9}}$, $\boldsymbol{g} = \boldsymbol{\mu_{13}}$, (b1) $q_+$, $\boldsymbol{f} = \boldsymbol{\mu_{3}}$, $\boldsymbol{g} = \boldsymbol{\mu_{13}}$, (b2) $q_+$, $\boldsymbol{f} = \boldsymbol{\mu_{1}}$, $\boldsymbol{g} = \boldsymbol{\mu_{11}}$, (c1) $q_-$, $\boldsymbol{f} = \boldsymbol{\mu_{10}}$, $\boldsymbol{g} = \boldsymbol{\mu_{12}}$, (c2) $q_-$, $\boldsymbol{f} = \boldsymbol{\mu_{8}}$, $\boldsymbol{g} = \boldsymbol{\mu_{11}}$, (d1) $q_-$, $\boldsymbol{f} = \boldsymbol{\mu_{11}}$, (d2) $q_+$, $\boldsymbol{f} = \boldsymbol{\mu_{4}}$, $\boldsymbol{g} = \boldsymbol{\mu_{9}}$, (e1) $q_-$, $\boldsymbol{f} = \boldsymbol{\mu_{1}}$, $\boldsymbol{g} = \boldsymbol{\mu_{10}}$, and (e2) $q_-$, $\boldsymbol{f} = \boldsymbol{\mu_{8}}$, $\boldsymbol{g} = \boldsymbol{\mu_{13}}$. Synthetic images (f-j) containing approx. color matching pixel blocks (block size: $32^2$ pixels) that correspond to previous stains, e.g. (f) simulates (a), (g) simulates (b), etc. The sample renditions are: (f1) $q_+$, $\boldsymbol{f} = \boldsymbol{\mu_{5}}$, (g1) $q_+$, $\boldsymbol{f} = \boldsymbol{\mu_{3}}$, $\boldsymbol{g} = \boldsymbol{\mu_{13}}$, (h1) $q_-$, $\boldsymbol{f} = \boldsymbol{\mu_{10}}$, $\boldsymbol{g} = \boldsymbol{\mu_{12}}$, (i1) $q_-$, $\boldsymbol{f} = \boldsymbol{\mu_{11}}$, and (j1) $q_-$, $\boldsymbol{f} = \boldsymbol{\mu_{1}}$, $\boldsymbol{g} = \boldsymbol{\mu_{10}}$. Synthetic images contain gray pixel blocks to simulate the microscopy slide backdrop. Pure unit quaternions $\boldsymbol{\mu_{1}}$-$\boldsymbol{\mu_{13}}$ are shown in Fig. \ref{['fig:m1m13']}. Images have $250^2$ pixels.