Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

One model to use them all: Training a segmentation model with complementary datasets

Alexander C. Jenke, Sebastian Bodenstedt, Fiona R. Kolbinger, Marius Distler, Jürgen Weitz, Stefanie Speidel

TL;DR

The paper tackles the data bottleneck in surgical scene segmentation by merging multiple partially labeled datasets that offer complementary annotations. It introduces implicit labeling, which leverages mutual exclusivity to infer negative samples from other classes and masks uncertain pixels, enabling a single model to learn from incomplete data. On the Dresden Surgical Anatomy Dataset, training a DeepLabV3 model with six classes as a unified output yields a 4.4 percentage point Dice improvement over an ensemble of per-class models and substantially reduces stomach–colon confusion by 24%, while achieving real-time inference. This approach demonstrates the feasibility of cross-dataset training for surgical scene segmentation, offering a practical path to scalable multi-class segmentation without requiring one large fully annotated dataset.

Abstract

Understanding a surgical scene is crucial for computer-assisted surgery systems to provide any intelligent assistance functionality. One way of achieving this scene understanding is via scene segmentation, where every pixel of a frame is classified and therefore identifies the visible structures and tissues. Progress on fully segmenting surgical scenes has been made using machine learning. However, such models require large amounts of annotated training data, containing examples of all relevant object classes. Such fully annotated datasets are hard to create, as every pixel in a frame needs to be annotated by medical experts and, therefore, are rarely available. In this work, we propose a method to combine multiple partially annotated datasets, which provide complementary annotations, into one model, enabling better scene segmentation and the use of multiple readily available datasets. Our method aims to combine available data with complementary labels by leveraging mutual exclusive properties to maximize information. Specifically, we propose to use positive annotations of other classes as negative samples and to exclude background pixels of binary annotations, as we cannot tell if they contain a class not annotated but predicted by the model. We evaluate our method by training a DeepLabV3 on the publicly available Dresden Surgical Anatomy Dataset, which provides multiple subsets of binary segmented anatomical structures. Our approach successfully combines 6 classes into one model, increasing the overall Dice Score by 4.4% compared to an ensemble of models trained on the classes individually. By including information on multiple classes, we were able to reduce confusion between stomach and colon by 24%. Our results demonstrate the feasibility of training a model on multiple datasets. This paves the way for future work further alleviating the need for one large, fully segmented datasets.

One model to use them all: Training a segmentation model with complementary datasets

TL;DR

The paper tackles the data bottleneck in surgical scene segmentation by merging multiple partially labeled datasets that offer complementary annotations. It introduces implicit labeling, which leverages mutual exclusivity to infer negative samples from other classes and masks uncertain pixels, enabling a single model to learn from incomplete data. On the Dresden Surgical Anatomy Dataset, training a DeepLabV3 model with six classes as a unified output yields a 4.4 percentage point Dice improvement over an ensemble of per-class models and substantially reduces stomach–colon confusion by 24%, while achieving real-time inference. This approach demonstrates the feasibility of cross-dataset training for surgical scene segmentation, offering a practical path to scalable multi-class segmentation without requiring one large fully annotated dataset.

Abstract

Understanding a surgical scene is crucial for computer-assisted surgery systems to provide any intelligent assistance functionality. One way of achieving this scene understanding is via scene segmentation, where every pixel of a frame is classified and therefore identifies the visible structures and tissues. Progress on fully segmenting surgical scenes has been made using machine learning. However, such models require large amounts of annotated training data, containing examples of all relevant object classes. Such fully annotated datasets are hard to create, as every pixel in a frame needs to be annotated by medical experts and, therefore, are rarely available. In this work, we propose a method to combine multiple partially annotated datasets, which provide complementary annotations, into one model, enabling better scene segmentation and the use of multiple readily available datasets. Our method aims to combine available data with complementary labels by leveraging mutual exclusive properties to maximize information. Specifically, we propose to use positive annotations of other classes as negative samples and to exclude background pixels of binary annotations, as we cannot tell if they contain a class not annotated but predicted by the model. We evaluate our method by training a DeepLabV3 on the publicly available Dresden Surgical Anatomy Dataset, which provides multiple subsets of binary segmented anatomical structures. Our approach successfully combines 6 classes into one model, increasing the overall Dice Score by 4.4% compared to an ensemble of models trained on the classes individually. By including information on multiple classes, we were able to reduce confusion between stomach and colon by 24%. Our results demonstrate the feasibility of training a model on multiple datasets. This paves the way for future work further alleviating the need for one large, fully segmented datasets.
Paper Structure (11 sections, 2 equations, 3 figures, 3 tables)

This paper contains 11 sections, 2 equations, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Methods
  3. Baselines
  4. Implication based labeling
  5. Metrics
  6. Evaluation
  7. Dataset
  8. Trials
  9. Results
  10. Discussion
  11. Conclusion

Figures (3)

  • Figure 1: A flowchart describing the flow of annotation data through the three architectures used in this work. The proposed implicit label generation is shown in the blue box.
  • Figure 2: Pixelwise confusion matrix per approach and trainset tested on the multi-class testset.
  • Figure 3: Segmentation results for 3 example images. The columns show the original frame, the ground truth, and the predicted segmentations of the fully supervised (FS), implicit labeling (IL), and ensemble (EN) approaches. Rows (a) and (b) show examples of good IL performance, (c) and (d) show more difficult cases.