Gaussian Random Fields as an Abstract Representation of Patient Metadata for Multimodal Medical Image Segmentation

TL;DR

This work tackles chronic wound segmentation and the challenge of incorporating patient metadata without compromising privacy. It introduces Gaussian Random Fields generated from metadata (DOB, gender, HDD) via spectral synthesis to create spatially structured representations that are merged with wound images in an early-fusion CNN pipeline using HarDNet-CWS. On a private Lancashire dataset with a DFUC 2022 test set, ensembles of GRF-based models yield small but consistent improvements in IoU ($+$0.0229) and DSC ($+$0.0220) over the baseline. The results suggest GRFs are a privacy-friendly, architecture-agnostic mechanism to infuse metadata into multimodal segmentation and may extend to broader tasks as more data become available.

Abstract

The growing rate of chronic wound occurrence, especially in patients with diabetes, has become a concerning trend in recent years. Chronic wounds are difficult and costly to treat, and have become a serious burden on health care systems worldwide. Chronic wounds can have devastating consequences for the patient, with infection often leading to reduced quality of life and increased mortality risk. Innovative deep learning methods for the detection and monitoring of such wounds have the potential to reduce the impact to both patient and clinician. We present a novel multimodal segmentation method which allows for the introduction of patient metadata into the training workflow whereby the patient data are expressed as Gaussian random fields. Our results indicate that the proposed method improved performance when utilising multiple models, each trained on different metadata categories. Using the Diabetic Foot Ulcer Challenge 2022 test set, when compared to the baseline results (intersection over union = 0.4670, Dice similarity coefficient = 0.5908) we demonstrate improvements of +0.0220 and +0.0229 for intersection over union and Dice similarity coefficient respectively. This paper presents the first study to focus on integrating patient data into a chronic wound segmentation workflow. Our results show significant performance gains when training individual models using specific metadata categories, followed by average merging of prediction masks using distance transforms. All source code for this study is available at: https://github.com/mmu-dermatology-research/multimodal-grf

Figures (5)

• Figure 1: Histogram analysis of the patient date of birth and patient health and disability decile metadata present in the multimodal dataset used in our experiments. Non-normal distribution is demonstrated in both types of data. Note that patient date of birth values are represented as timestamps.
• Figure 2: Illustration of the types of Gaussian random fields generated for use in our multimodal chronic wound segmentation experiments. Examples (a), (c), and (e) were generated with an $i$ value of 2, and examples (b), (d), and (f) were generated using an $i$ value of 5. The top row examples were generated using DOB, the second row examples were generated using gender, and the third row examples were generated using HDD.
• Figure 3: Illustration of the tensor merging operation when merging wound image RGB tensors with single channel GRF tensors to produce new 4D tensors. The three RGB channels represent the wound image features, while the GRF channel is an abstract visual representation of a patient metadata item.
• Figure 4: Illustration of the prediction mask average merging process completed using distance transforms: (a) shows the original wound image, (b) is a mask generated by the model trained using DOB GRFs, (c) is a mask generated by the model trained using gender GRFs, (d) is a mask generated by the model trained using HDD GRFs, and (e) is the average merged mask. Note that images have been cropped for illustrative purposes.
• Figure 5: Illustration showing four examples of where the averaged merged predictions from the $i = 2$ models show improved performance over the baseline results. Column (a) shows the original wound image, column (b) shows the ground truth mask, column (c) shows the baseline HarDNet-CWS prediction, and column (d) shows the averaged merged prediction for the $i = 2$ models.