Study design: Prospective, blinded interrater reliability study. Objective: To determine the interrater reliability of the modified Scapular Assistance Test (mSAT) with and without the use of additional handheld weights. Methods: 110 Shoulder patients with various shoulder pathologies were consecutively recruited. Tests were performed independently and randomly on each participant by 2 different examiners, which were blinded to further clinical information (e.g. patient history, former diagnostic results). Percent agreement, Cohen's kappa (Κ), proportion of positive/negative agreement, maximum Κ, prevalence and bias indexes and prevalence-adjusted-bias-adjusted kappa (PABAK) were calculated as estimates of interrater reliability of the mSAT with and without additional handheld weights. Weights were chosen according to body weight. Results: The reliability measures for the mSAT (Cohen's Κ: 0.68, confidence interval (CI): 0.51-0.85; PABAK: 0.78, CI: 0.67-0.90) as well as for the mSAT with handheld weights (Cohen's Κ: 0.63, CI: 0.44-0.81; PABAK: 0.76, CI: 0.64-0.88) showed substantial agreement according to the classification system proposed by Landis and Koch. Conclusions: Based on the results of this study, the mSAT with and without additional weights can be considered as reliable for clinical use. Since both tests showed substantial agreement, the use of additional handheld weights might not be necessary in case of obvious scapula dyskinesis. However, to perform the mSAT with/without additional weights should depend not only on its reliability values.Name of the public trials registry and the registration number: German Clinical Trials Register, protocol number DRKS00005377.