A single pair of helical edge states as realized at the boundary of a quantum spin Hall insulator is known to be robust against elastic single particle backscattering as long as time reversal symmetry is preserved. However, there is no symmetry preventing inelastic backscattering as brought about by phonons in the presence of Rashba spin orbit coupling. In this Letter, we show that the quantized conductivity of a single channel of helical Dirac electrons is protected even against this inelastic mechanism to leading order. We further demonstrate that this result remains valid when Coulomb interaction is included in the framework of a helical Tomonaga Luttinger liquid.