Evaluating prompt strategies for LLMBased de-identification of german discharge letters: A feasibility study using grascco

Research output: Contribution to journal › Conference article › Contributed › peer-review

Abstract

Introduction: Clinical text de-identification is essential for secondary healthcare data use but remains challenging due to heterogeneous documentation. Methods: We evaluated LLM-based de-identification of synthetic German discharge letters (GraSCCo) using four prompting strategies with GPT-4o and an additional GPT-OSS comparison. Performance was assessed via precision, recall, F1-score, false positives, and relative text reduction. Results: The baseline setup was not suitable due to excessive text loss. F1-scores reached 0.81 (session-isolated), 0.79 (structured-input), 0.93 (optimized prompt, GPT-4o), and 0.90 (GPT-OSS). Discussion: Results indicate that prompt refinement has a stronger impact on de-identification quality than structural preprocessing. LLM-based de-identification is feasible but requires careful prompt design and validation on real-world data.

Details

Original languageEnglish
Pages (from-to)26-30
Number of pages5
JournalStudies in health technology and informatics
Publication statusPublished - 29 Jun 2026
Peer-reviewedYes

Conference

Title24th International Conference on Informatics, Management, and Technology in Healthcare
SubtitleHealth Sciences Informatics Leads and Empowers the Digital Health Transformation
Abbreviated titleICIMTH 2026
Conference number24
Duration3 - 5 July 2026
Website
LocationDivani Palace Acropolis
CityAthens
CountryGreece

External IDs

PubMed 42393951
ORCID /0000-0002-1887-4772/work/222763325
ORCID /0000-0002-9888-8460/work/222763649

Keywords

Keywords

  • Clinical NLP, De-identification, LLMs, Prompt engineering