We show that the Klein bottle entropy [H.-H. Tu, Phys. Rev. Lett. 119, 261603 (2017)PRLTAO0031-900710.1103/PhysRevLett.119.261603] for conformal field theories perturbed by a relevant operator is a universal function of the dimensionless coupling constant. The universal scaling of the Klein bottle entropy near criticality provides an efficient approach to extract the scaling dimension of lattice operators via data collapse. As paradigmatic examples, we validate the universal scaling of the Klein bottle entropy for Ising and Z3 parafermion conformal field theories with various perturbations using numerical simulation with continuous matrix product operator approach.