Evaluation of a clinical decision support system for rare diseases: a qualitative study

Publikation: Beitrag in FachzeitschriftForschungsartikelBeigetragenBegutachtung


  • Jannik Schaaf - , Medical Informatics Group (MIG), Universitätsklinikum Frankfurt (Autor:in)
  • Martin Sedlmayr - , Institut für Medizinische Informatik und Biometrie, Universitätsklinikum Carl Gustav Carus Dresden, Technische Universität Dresden (Autor:in)
  • Brita Sedlmayr - , Institut für Medizinische Informatik und Biometrie, Universitätsklinikum Carl Gustav Carus Dresden, Technische Universität Dresden (Autor:in)
  • Hans-Ulrich Prokosch - , Friedrich-Alexander-Universität Erlangen-Nürnberg (Autor:in)
  • Holger Storf - , Universitätsklinikum Frankfurt (Autor:in)


BACKGROUND: Rare Diseases (RDs) are difficult to diagnose. Clinical Decision Support Systems (CDSS) could support the diagnosis for RDs. The Medical Informatics in Research and Medicine (MIRACUM) consortium developed a CDSS for RDs based on distributed clinical data from eight German university hospitals. To support the diagnosis for difficult patient cases, the CDSS uses data from the different hospitals to perform a patient similarity analysis to obtain an indication of a diagnosis. To optimize our CDSS, we conducted a qualitative study to investigate usability and functionality of our designed CDSS.

METHODS: We performed a Thinking Aloud Test (TA-Test) with RDs experts working in Rare Diseases Centers (RDCs) at MIRACUM locations which are specialized in diagnosis and treatment of RDs. An instruction sheet with tasks was prepared that the participants should perform with the CDSS during the study. The TA-Test was recorded on audio and video, whereas the resulting transcripts were analysed with a qualitative content analysis, as a ruled-guided fixed procedure to analyse text-based data. Furthermore, a questionnaire was handed out at the end of the study including the System Usability Scale (SUS).

RESULTS: A total of eight experts from eight MIRACUM locations with an established RDC were included in the study. Results indicate that more detailed information about patients, such as descriptive attributes or findings, can help the system perform better. The system was rated positively in terms of functionality, such as functions that enable the user to obtain an overview of similar patients or medical history of a patient. However, there is a lack of transparency in the results of the CDSS patient similarity analysis. The study participants often stated that the system should present the user with an overview of exact symptoms, diagnosis, and other characteristics that define two patients as similar. In the usability section, the CDSS received a score of 73.21 points, which is ranked as good usability.

CONCLUSIONS: This qualitative study investigated the usability and functionality of a CDSS of RDs. Despite positive feedback about functionality of system, the CDSS still requires some revisions and improvement in transparency of the patient similarity analysis.


FachzeitschriftBMC medical informatics and decision making
PublikationsstatusVeröffentlicht - 18 Feb. 2021

Externe IDs

PubMedCentral PMC7890997
Scopus 85101222154
ORCID /0000-0002-9888-8460/work/142254087



  • Decision Support Systems, Clinical, Hospitals, Humans, Qualitative Research, Rare Diseases/diagnosis