Evaluating prompt strategies for LLMBased de-identification of german discharge letters: A feasibility study using grascco
Research output: Contribution to journal › Conference article › Contributed › peer-review
Contributors
Abstract
Introduction: Clinical text de-identification is essential for secondary healthcare data use but remains challenging due to heterogeneous documentation. Methods: We evaluated LLM-based de-identification of synthetic German discharge letters (GraSCCo) using four prompting strategies with GPT-4o and an additional GPT-OSS comparison. Performance was assessed via precision, recall, F1-score, false positives, and relative text reduction. Results: The baseline setup was not suitable due to excessive text loss. F1-scores reached 0.81 (session-isolated), 0.79 (structured-input), 0.93 (optimized prompt, GPT-4o), and 0.90 (GPT-OSS). Discussion: Results indicate that prompt refinement has a stronger impact on de-identification quality than structural preprocessing. LLM-based de-identification is feasible but requires careful prompt design and validation on real-world data.
Details
| Original language | English |
|---|---|
| Pages (from-to) | 26-30 |
| Number of pages | 5 |
| Journal | Studies in health technology and informatics |
| Publication status | Published - 29 Jun 2026 |
| Peer-reviewed | Yes |
Conference
| Title | 24th International Conference on Informatics, Management, and Technology in Healthcare |
|---|---|
| Subtitle | Health Sciences Informatics Leads and Empowers the Digital Health Transformation |
| Abbreviated title | ICIMTH 2026 |
| Conference number | 24 |
| Duration | 3 - 5 July 2026 |
| Website | |
| Location | Divani Palace Acropolis |
| City | Athens |
| Country | Greece |
External IDs
| PubMed | 42393951 |
|---|---|
| ORCID | /0000-0002-1887-4772/work/222763325 |
| ORCID | /0000-0002-9888-8460/work/222763649 |
Keywords
ASJC Scopus subject areas
Keywords
- Clinical NLP, De-identification, LLMs, Prompt engineering