Exploring semantic consistency in unpaired image translation to generate data for surgical applications

Research output: Contribution to journalResearch articleContributedpeer-review

Abstract

Purpose: In surgical computer vision applications, data privacy and expert annotation challenges impede the acquisition of labeled training data. Unpaired image-to-image translation techniques have been explored to automatically generate annotated datasets by translating synthetic images into a realistic domain. The preservation of structure and semantic consistency, i.e., per-class distribution during translation, poses a significant challenge, particularly in cases of semantic distributional mismatch. Method: This study empirically investigates various translation methods for generating data in surgical applications, explicitly focusing on semantic consistency. Through our analysis, we introduce a novel and simple combination of effective approaches, which we call ConStructS. The defined losses within this approach operate on multiple image patches and spatial resolutions during translation. Results: Various state-of-the-art models were extensively evaluated on two challenging surgical datasets. With two different evaluation schemes, the semantic consistency and the usefulness of the translated images on downstream semantic segmentation tasks were evaluated. The results demonstrate the effectiveness of the ConStructS method in minimizing semantic distortion, with images generated by this model showing superior utility for downstream training. Conclusion: In this study, we tackle semantic inconsistency in unpaired image translation for surgical applications with minimal labeled data. The simple model (ConStructS) enhances consistency during translation and serves as a practical way of generating fully labeled and semantically consistent datasets at minimal cost. Our code is available at https://gitlab.com/nct_tso_public/constructs.

Details

Original languageEnglish
Pages (from-to)985-993
Number of pages9
JournalInternational journal of computer assisted radiology and surgery
Volume19
Issue number6
Early online date26 Feb 2024
Publication statusPublished - Jun 2024
Peer-reviewedYes

External IDs

dblp journals/cars/VenkateshRPKDWS24
Mendeley ac407c1f-bab3-3490-af6e-592f86a434d2
ORCID /0000-0002-4590-1908/work/163294055