Organic solar cells demonstrate external quantum eciencies and fill factors approaching those of conventional photovoltaic technologies. However, as compared with the optical gap of the absorber materials, their open-circuit voltage is much lower, largely due to the presence of significant non-radiative recombination. Here, we study a large data set of published and new material combinations and find that non-radiative voltage losses decrease with increasing charge-transfer-state energies. This observation is explained by considering non-radiative charge-transfer-state decay as electron transfer in the Marcus inverted regime, being facilitated by a common skeletal molecular vibrational mode. Our results suggest an intrinsic link between non-radiative voltage losses and electron-vibration coupling, indicating that these losses are unavoidable. Accordingly, the theoretical upper limit for the power conversion effciency of single-junction organic solar cells would be reduced to about 25.5% and the optimal optical gap increases to (1.45-1.65) eV, that is, (0.20.3) eV higher than for technologies with minimized non-radiative voltage losses.