Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Multi-View Consistent Wound Segmentation With Neural Fields

Remi Chierchia, Léo Lebrat, David Ahmedt-Aristizabal, Yulia Arzhaeva, Olivier Salvado, Clinton Fookes, Rodrigo Santa Cruz

TL;DR

This work tackles the challenge of obtaining view-consistent 3D wound segmentation from multi-view images by introducing WoundNeRF, a NeRF-based framework that learns a 3D semantic field through a geometry MLP with an SDF and a semantic head. It aggregates cross-view 2D segmentations into a 3D representation and optimizes with a weighted cross-entropy loss, using a two-stage training to stabilize learning. The method outperforms 2D transformer-based segmentation and traditional 3D/2D fusion on 73 videos, delivering smoother boundaries, higher recall, and robust 3D coherence, while also enabling wound mesh extraction for documentation. This approach offers a principled route to view-consistent wound assessment with potential to improve clinical documentation and tracking of healing progress, and sets the stage for confidence-informed segmentation in medical imaging.

Abstract

Wound care is often challenged by the economic and logistical burdens that consistently afflict patients and hospitals worldwide. In recent decades, healthcare professionals have sought support from computer vision and machine learning algorithms. In particular, wound segmentation has gained interest due to its ability to provide professionals with fast, automatic tissue assessment from standard RGB images. Some approaches have extended segmentation to 3D, enabling more complete and precise healing progress tracking. However, inferring multi-view consistent 3D structures from 2D images remains a challenge. In this paper, we evaluate WoundNeRF, a NeRF SDF-based method for estimating robust wound segmentations from automatically generated annotations. We demonstrate the potential of this paradigm in recovering accurate segmentations by comparing it against state-of-the-art Vision Transformer networks and conventional rasterisation-based algorithms. The code will be released to facilitate further development in this promising paradigm.

Multi-View Consistent Wound Segmentation With Neural Fields

TL;DR

This work tackles the challenge of obtaining view-consistent 3D wound segmentation from multi-view images by introducing WoundNeRF, a NeRF-based framework that learns a 3D semantic field through a geometry MLP with an SDF and a semantic head. It aggregates cross-view 2D segmentations into a 3D representation and optimizes with a weighted cross-entropy loss, using a two-stage training to stabilize learning. The method outperforms 2D transformer-based segmentation and traditional 3D/2D fusion on 73 videos, delivering smoother boundaries, higher recall, and robust 3D coherence, while also enabling wound mesh extraction for documentation. This approach offers a principled route to view-consistent wound assessment with potential to improve clinical documentation and tracking of healing progress, and sets the stage for confidence-informed segmentation in medical imaging.

Abstract

Wound care is often challenged by the economic and logistical burdens that consistently afflict patients and hospitals worldwide. In recent decades, healthcare professionals have sought support from computer vision and machine learning algorithms. In particular, wound segmentation has gained interest due to its ability to provide professionals with fast, automatic tissue assessment from standard RGB images. Some approaches have extended segmentation to 3D, enabling more complete and precise healing progress tracking. However, inferring multi-view consistent 3D structures from 2D images remains a challenge. In this paper, we evaluate WoundNeRF, a NeRF SDF-based method for estimating robust wound segmentations from automatically generated annotations. We demonstrate the potential of this paradigm in recovering accurate segmentations by comparing it against state-of-the-art Vision Transformer networks and conventional rasterisation-based algorithms. The code will be released to facilitate further development in this promising paradigm.
Paper Structure (9 sections, 3 equations, 4 figures, 2 tables)

This paper contains 9 sections, 3 equations, 4 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Method
  3. Optimization Strategy
  4. Implementation Details
  5. Experiments and Results
  6. Discussion
  7. Conclusion
  8. Compliance with ethical standards
  9. Acknowledgments

Figures (4)

  • Figure 1: View inconsistency from mapping 2D expert-annotated masks from different viewpoints onto the underlying 3D surface. Overlapping regions are highlighted in magenta, while non-overlapping regions indicate disagreement.
  • Figure 2: WoundNeRF's pipeline. For clarity, the explicit dependence on input variables is omitted.
  • Figure 3: Qualitative comparison of segmentation masks across the compared methods for four wounds. Our method additionally renders the learned RGB appearance, visible in the background region with a subtle colour tint and slightly reduced sharpness. The top and bottom rows display wound bed (1) masks, while the middle rows represent the two most common tissue classes, granulation (2) and slough (4), respectively. The necrotic (3) and unknown (5) tissue classes are also displayed, except for the epithelial class.
  • Figure 4: Qualitative comparison 2D predictions versus our method trained with few views. The first two rows present three different views, showing the 2D predictions on the left and renderings produced by our method on the right. The bottom row displays the corresponding wound segmentation on the 3D mesh extracted from our method.