Reporting checklist for foundation and large language models in medical research (REFINE): an international consensus guideline

Research output: Contribution to journalResearch articleContributedpeer-review

Contributors

  • Ismail Mese - , Uskudar State Hospital (Author)
  • Tugba Akinci D'Antonoli - , University Children's Hospital of Basel (Author)
  • Christian Bluethgen - , Stanford University (Author)
  • Keno Bressem - , Klinikum Rechts der Isar (MRI TUM) (Author)
  • Renato Cuocolo - , University of Salerno (Author)
  • Akshay Chaudhari - , Stanford University (Author)
  • Ali S Tejani - , University of California at San Francisco (Author)
  • Amanda Isaac - , King's College London (KCL) (Author)
  • Andrea Ponsiglione - , Universita' di Napoli Federico II (Author)
  • Aymen Meddeb - , Charité – Universitätsmedizin Berlin (Author)
  • Bardia Khosravi - , Mayo Clinic Rochester, MN (Author)
  • Bastien Le Guellec - , University Hospital of Lille (Author)
  • Charles E Kahn - , University of Pennsylvania (Author)
  • Chong Hyun Suh - , Asan Medical Center (Author)
  • Daniel Pinto Dos Santos - , University Medical Center Mainz (Author)
  • Dow-Mu Koh - , Institute of Cancer Research (Author)
  • Eleftherios Tzanis - , University of Crete (Author)
  • Elmar Kotter - , University Medical Center Freiburg (Author)
  • Errol Colak - , University of Toronto (Author)
  • Felipe Kitamura - , Federal University of São Paulo (Author)
  • Felix Busch - , Klinikum Rechts der Isar (MRI TUM) (Author)
  • Felix Nensa - , University Hospital Essen (Author)
  • Guang Yang - , East China Normal University (Author)
  • Henning Müller - , The Sense Innovation & Research Center (Author)
  • Jakob Nikolas Kather - , Department of Internal Medicine I, Else Kröner Fresenius Center for Digital Health, National Center for Tumor Diseases (NCT) Heidelberg, Leeds Teaching Hospitals NHS Trust (Author)
  • Jawed Nawabi - , Charité – Universitätsmedizin Berlin (Author)
  • Jens Kleesiek - , University Hospital Essen (Author)
  • Jingyu Zhong - , Shanghai Jiao Tong University (Author)
  • João Santinha - , University of Lisbon (Author)
  • Johannes Haubold - , University Hospital Essen (Author)
  • José Guilherme de Almeida - , Champalimaud Foundation (Author)
  • Karim Lekadir - , ICREA - Catalan Institution for Research and Advanced Studies (Author)
  • Kostas Marias - , Foundation for Research and Technology-Hellas (Author)
  • Lara Noelle Reiner - , Charité – Universitätsmedizin Berlin (Author)
  • Lena Maier-Hein - , Mohamed Bin Zayed University of Artificial Intelligence (Author)
  • Linda Moy - , NYU Grossman School of Medicine (Author)
  • Lisa C Adams - , Klinikum Rechts der Isar (MRI TUM) (Author)
  • Luis Martí-Bonmatí - , La Fe University and Polytechnic Hospital (Author)
  • Magdalini Paschali - , Stanford University (Author)
  • Mana Moassefi - , Mayo Clinic Rochester, MN (Author)
  • Matthias Dietzel - , University Hospital at the Friedrich-Alexander University Erlangen-Nürnberg (Author)
  • Merel Huisman - , Radboud University Medical Center (Author)
  • Michael Ingrisch - , Konrad Zuse School of Excellence in Reliable AI (Author)
  • Michail E Klontzas - , Foundation for Research and Technology-Hellas (Author)
  • Nikolaos Papanikolaou - , Royal Marsden NHS Foundation Trust (Author)
  • Oliver Diaz - , University of Barcelona (Author)
  • Paulo Kuriki - , University of Texas Southwestern Medical Center (Author)
  • Philipp Seeböck - , Medical University of Vienna (Author)
  • Pouria Rouzrokh - , Mayo Clinic Rochester, MN (Author)
  • Quirin D Strotzer - , University Hospital Regensburg (Author)
  • Seong Ho Park - , Asan Medical Center (Author)
  • Shahriar Faghani - , Mayo Clinic Rochester, MN (Author)
  • Soroosh Tayebi Arasteh - , Stanford Medicine (Author)
  • Su Hwan Kim - , Klinikum Rechts der Isar (MRI TUM) (Author)
  • Vasantha Kumar Venugopal - , Rajiv Gandhi Cancer Institute and Research Centre (Author)
  • Woojin Kim - , VA Palo Alto Health Care System (VAPAHCS) (Author)
  • Burak Kocak - , Başakşehir Çam And Sakura Cıty Hospıtal (Author)

Abstract

PURPOSE: To develop the REporting checklist for FoundatIon and large laNguagE models (REFINE), an international reporting guideline for transparent and reproducible reporting of foundation model (FM) and large language model (LLM) studies in medical research, including imaging artificial intelligence (AI) applications.

METHODS: The protocol was prespecified and publicly archived. A modified Delphi process was conducted to establish reporting standards for unimodal and multimodal FM and LLM applications involving text, imaging, and structured data. The steering committee coordinated protocol development, expert recruitment, all Delphi rounds, and the harmonization phase. Decisions were made based on predefined consensus thresholds. In Rounds 1 and 2, structured ratings and free-text feedback informed iterative revisions. In the post-Delphi harmonization phase, terminology was standardized, and detailed reporting instructions were finalized.

RESULTS: The REFINE development group comprised 57 contributors from 17 countries, and 54 panelists from 16 countries completed Rounds 1 and 2. The harmonization phase was completed by three expert panelists and the steering committee. The entire process produced a 44-item, six-section framework with standardized terminology and detailed reporting instructions, supported by an online platform for practical use (https://refinechecklist.github.io/refine/checklist.html).

CONCLUSION: The REFINE provides a comprehensive, consensus-based reporting standard for medical FM and LLM research, including imaging AI studies. The online version facilitates practical implementation.

CLINICAL SIGNIFICANCE: The REFINE enables transparent, comparable, and reproducible reporting of FM and LLM studies, supporting reliable evidence synthesis in medical and imaging-focused AI studies.

Details

Original languageEnglish
JournalDiagnostic and interventional radiology : official journal of the Turkish Society of Radiology
Publication statusE-pub ahead of print - 26 Feb 2026
Peer-reviewedYes

External IDs

ORCID /0000-0002-3730-5348/work/212492326
unpaywall 10.4274/dir.2026.263812

Keywords