Background Medical communication is a core task of physicians, and its quality affects both patients and physicians. Hence, the teaching and measurement of medical communication competence (MCC) are crucial during medical studies. This study aims to explore the factorial and construct validity of the Video-Based Assessment of Medical Communication Competence (VA-MeCo), an online-situational judgement test (SJT) designed to measure three aspects of medical students’ communication competence in patient encounters: advancing content, providing structure, and building relationship. Methods We conducted an online survey with N= 395 medical students. Factorial validity was tested by confirmatory factor analysis (CFA). To investigate convergent and discriminant aspects of construct validity, we tested correlations of participants’ VA-MeCo sub-test scores with scores in relevant cognitive variables, patient-interaction, and general personality traits. Results The CFA confirmed the expected three-dimensional factorial structure showing good model fit and highly correlated dimensions. A McDonald’s Ω of.94 for the complete test and >.81 for the sub-scales indicated high reliability. Regarding construct validity, the directions of the correlations were in line with the theoretically assumed associations; correlation sizes partly deviated from our expectations. Conclusions The results support that MCC can be validly measured using the VA-MeCo. The CFA results align with theory and previous studies that proposed three distinguishable dimensions of MCC. The findings on convergent and discriminant validity demonstrate that the test measures MCC as a specific construct that is positively related to patient-interaction; moreover, it can be distinguished from related but more generic constructs (e.g., social competence) and is not overly confounded with construct-irrelevant traits (e.g., intelligence). Notwithstanding the need for further validation, the results indicate that the VA-MeCo is a useful test for assessing MCC in medical education.