Purpose: Clinical validation of “BrainLossNet”, a deep learning-based method for fast and robust estimation of brain volume loss (BVL) from longitudinal T1-weighted MRI, for the detection of accelerated BVL in multiple sclerosis (MS) and for the discrimination between MS patients with versus without disability progression. Materials and methods: A longitudinal normative database of healthy controls (n = 563), two mono-scanner MS cohorts (n = 414, 156) and a mixed-scanner cohort acquired for various indications (n = 216) were included retrospectively. Mean observation period from the baseline (BL) to the last follow-up (FU) MRI scan was 2–3 years. Expanded Disability Status Scale (EDSS) at BL and FU was available in 149 MS patients. Annual BVL was computed using BrainLossNet and Siena and then adjusted for age. Repeated-measures ANOVA and Cohen's effect size were used to compare BrainLossNet and Siena regarding the detection of accelerated BVL and the differentiation between MS patients with versus without EDSS progression. Results: Cohen's effect size for the differentiation of patients from healthy controls based on the age-adjusted annual BVL was larger with BrainLossNet than with Siena (MS cohort 1: 0.927 versus 0.495, MS cohort 2: 0.671 versus 0.456, mixed-scanner cohort: 0.918 versus 0.730, all p < 0.001). Cohen's effect size for the discrimination between MS patients with (n = 51) versus without (n = 98) EDSS progression was larger with BrainLossNet (0.503 versus 0.400, p = 0.048). Conclusion: BrainLossNet can provide added value in clinical routine and MS therapy trials regarding the detection of accelerated BVL in MS and the differentiation between MS patients with versus without disability progression.