Technical metrics used to evaluate medical artificial intelligence tools often fail to predict their clinical impact. We characterize this discordance and propose a framework of study designs to guide the translational process for clinical artificial intelligence tools, acknowledging their diversity and specific validation requirements.