One model to use them all: training a segmentation model with complementary datasets

Research output: Contribution to journalResearch articleContributedpeer-review

Contributors

Abstract

PURPOSE: Understanding surgical scenes is crucial for computer-assisted surgery systems to provide intelligent assistance functionality. One way of achieving this is via scene segmentation using machine learning (ML). However, such ML models require large amounts of annotated training data, containing examples of all relevant object classes, which are rarely available. In this work, we propose a method to combine multiple partially annotated datasets, providing complementary annotations, into one model, enabling better scene segmentation and the use of multiple readily available datasets.

METHODS: Our method aims to combine available data with complementary labels by leveraging mutual exclusive properties to maximize information. Specifically, we propose to use positive annotations of other classes as negative samples and to exclude background pixels of these binary annotations, as we cannot tell if a positive prediction by the model is correct.

RESULTS: We evaluate our method by training a DeepLabV3 model on the publicly available Dresden Surgical Anatomy Dataset, which provides multiple subsets of binary segmented anatomical structures. Our approach successfully combines 6 classes into one model, significantly increasing the overall Dice Score by 4.4% compared to an ensemble of models trained on the classes individually. By including information on multiple classes, we were able to reduce the confusion between classes, e.g. a 24% drop for stomach and colon.

CONCLUSION: By leveraging multiple datasets and applying mutual exclusion constraints, we developed a method that improves surgical scene segmentation performance without the need for fully annotated datasets. Our results demonstrate the feasibility of training a model on multiple complementary datasets. This paves the way for future work further alleviating the need for one specialized large, fully segmented dataset but instead the use of already existing datasets.

Details

Original languageEnglish
Pages (from-to)1233-1241
Number of pages9
JournalInternational journal of computer assisted radiology and surgery
Volume19
Issue number6
Early online date27 Apr 2024
Publication statusPublished - Jun 2024
Peer-reviewedYes

External IDs

Scopus 85191718442
dblp journals/cars/JenkeBKDWS24
Mendeley 68634d01-2179-3497-8cbe-9703d9e83dbc
ORCID /0000-0002-4590-1908/work/163294145
ORCID /0000-0002-4675-417X/work/170587566

Keywords

Keywords

  • Databases, Factual, Datasets as Topic, Humans, Image Processing, Computer-Assisted/methods, Machine Learning, Surgery, Computer-Assisted/methods