Evaluating prompt strategies for LLMBased de-identification of german discharge letters: A feasibility study using grascco

Publikation: Beitrag in Fachzeitschrift › Konferenzartikel › Beigetragen › Begutachtung

Abstract

Introduction: Clinical text de-identification is essential for secondary healthcare data use but remains challenging due to heterogeneous documentation. Methods: We evaluated LLM-based de-identification of synthetic German discharge letters (GraSCCo) using four prompting strategies with GPT-4o and an additional GPT-OSS comparison. Performance was assessed via precision, recall, F1-score, false positives, and relative text reduction. Results: The baseline setup was not suitable due to excessive text loss. F1-scores reached 0.81 (session-isolated), 0.79 (structured-input), 0.93 (optimized prompt, GPT-4o), and 0.90 (GPT-OSS). Discussion: Results indicate that prompt refinement has a stronger impact on de-identification quality than structural preprocessing. LLM-based de-identification is feasible but requires careful prompt design and validation on real-world data.

Details

OriginalspracheEnglisch
Seiten (von - bis)26-30
Seitenumfang5
FachzeitschriftStudies in health technology and informatics
PublikationsstatusVeröffentlicht - 29 Juni 2026
Peer-Review-StatusJa

Konferenz

Titel24th International Conference on Informatics, Management, and Technology in Healthcare
UntertitelHealth Sciences Informatics Leads and Empowers the Digital Health Transformation
KurztitelICIMTH 2026
Veranstaltungsnummer24
Dauer3 - 5 Juli 2026
Webseite
OrtDivani Palace Acropolis
StadtAthens
LandGriechenland

Externe IDs

PubMed 42393951
ORCID /0000-0002-1887-4772/work/222763325
ORCID /0000-0002-9888-8460/work/222763649

Schlagworte

ASJC Scopus Sachgebiete

Schlagwörter

  • Clinical NLP, De-identification, LLMs, Prompt engineering