Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Tree-based Semantic Losses: Application to Sparsely-supervised Large Multi-class Hyperspectral Segmentation

Junwen Wang, Oscar Maccormac, William Rochford, Aaron Kujawa, Jonathan Shapey, Tom Vercauteren

TL;DR

This work tackles sparse, multi-class segmentation in hyperspectral surgical imaging by exploiting a predefined label hierarchy through two semantically informed losses. It introduces a Wasserstein-based loss and a tree-based semantic cross-entropy loss that leverage hierarchical label structure, and integrates them with an OOD-aware, sparse-annotation training framework. On a 107-class dataset gathered from neurosurgical procedures, the proposed losses achieve state-of-the-art performance for both top-level and leaf-node segmentation, while enabling pixel-level OOD detection without harming in-distribution accuracy. Across extensive ablations, the authors show that top-level hierarchical weighting substantially influences performance and that the methods yield more semantically coherent distinctions between normal and abnormal tissues, with clear qualitative and quantitative gains.

Abstract

Hyperspectral imaging (HSI) shows great promise for surgical applications, offering detailed insights into biological tissue differences beyond what the naked eye can perceive. Refined labelling efforts are underway to train vision systems to distinguish large numbers of subtly varying classes. However, commonly used learning methods for biomedical segmentation tasks penalise all errors equivalently and thus fail to exploit any inter-class semantics in the label space. In this work, we introduce two tree-based semantic loss functions which take advantage of a hierarchical organisation of the labels. We further incorporate our losses in a recently proposed approach for training with sparse, background-free annotations. Extensive experiments demonstrate that our proposed method reaches state-of-the-art performance on a sparsely annotated HSI dataset comprising $107$ classes organised in a clinically-defined semantic tree structure. Furthermore, our method enables effective detection of out-of-distribution (OOD) pixels without compromising segmentation performance on in-distribution (ID) pixels.

Tree-based Semantic Losses: Application to Sparsely-supervised Large Multi-class Hyperspectral Segmentation

TL;DR

This work tackles sparse, multi-class segmentation in hyperspectral surgical imaging by exploiting a predefined label hierarchy through two semantically informed losses. It introduces a Wasserstein-based loss and a tree-based semantic cross-entropy loss that leverage hierarchical label structure, and integrates them with an OOD-aware, sparse-annotation training framework. On a 107-class dataset gathered from neurosurgical procedures, the proposed losses achieve state-of-the-art performance for both top-level and leaf-node segmentation, while enabling pixel-level OOD detection without harming in-distribution accuracy. Across extensive ablations, the authors show that top-level hierarchical weighting substantially influences performance and that the methods yield more semantically coherent distinctions between normal and abnormal tissues, with clear qualitative and quantitative gains.

Abstract

Hyperspectral imaging (HSI) shows great promise for surgical applications, offering detailed insights into biological tissue differences beyond what the naked eye can perceive. Refined labelling efforts are underway to train vision systems to distinguish large numbers of subtly varying classes. However, commonly used learning methods for biomedical segmentation tasks penalise all errors equivalently and thus fail to exploit any inter-class semantics in the label space. In this work, we introduce two tree-based semantic loss functions which take advantage of a hierarchical organisation of the labels. We further incorporate our losses in a recently proposed approach for training with sparse, background-free annotations. Extensive experiments demonstrate that our proposed method reaches state-of-the-art performance on a sparsely annotated HSI dataset comprising classes organised in a clinically-defined semantic tree structure. Furthermore, our method enables effective detection of out-of-distribution (OOD) pixels without compromising segmentation performance on in-distribution (ID) pixels.

Paper Structure

This paper contains 14 sections, 7 equations, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Material and methods
  3. Surgical imaging dataset
  4. Wasserstein distance in label space
  5. Tree-based semantic cross-entropy loss
  6. Out-of-distribution segmentation
  7. Experimental results
  8. Implementation details
  9. Choice of tree weights / ground distances
  10. Cross-validation results
  11. Error analysis
  12. Qualitative results
  13. Conclusion
  14. Disclosure of Interests.

Figures (3)

  • Figure 1: Full tree-based label hierarchy of the surgical HSI dataset. From top to bottom, the hierarchy progresses from coarse object categories to specific classes. The colour coding matches the ground-truth mask at each level.
  • Figure 2: Confusion matrices on top-level nodes. Results are averaged across all cross-validation folds. Class names from top-left to bottom-right: Other, Out-of-focus Area, Vascular, Normal Tissue, Abnormal Tissue and Surgical Equipment.
  • Figure 3: Qualitative result on top-level classes. We show the result of same image using different methods at confidence threshold $\tau_m$. Baseline results at $\tau_0=0$ are added to represent result without outlier detection.