A large and oscillating spin Hall effect (SHE) in organic polymer ladders originating from polaron transport has been proposed in a previous work [Hu, Phys. Rev. B 106, 144309 (2022)]10.1103/PhysRevB.106.144309, where a new mechanism of skew scattering off transient deformations of organic lattices caused by electron-lattice coupling has been revealed. In order to elucidate the origin of skew scattering, we analyze the components of the lattice distortions and investigate their roles in the SHE in the framework of the Su-Schrieffer-Heeger (SSH) model and the nonadiabatic dynamics method. By introducing a damping term into Newton's equation of motion, the breathers, i.e., lattice vibrations, generated by passing polarons and fueled by excess energy ultimately provided by the electric field, and charge-induced self-trapped distortions associated with the polarons themselves, are distinguished. A significant contribution to the SHE from skew scattering off the breathers is found, where the contributed percentage is related to the interchain coupling strength. Employing spectral analysis, we find that the effect of the breathers on the oscillation behavior mainly appears at low frequencies by modifying the electron transfer between chains. Moreover, by manipulating the amplitude of the breathers using a stronger field or shorter chains, the amplitude of the SHE may be controlled and enhanced. This work deepens our understanding of the microscopic mechanism of the SHE in organic polymers, and sheds light on the importance of spin-orbit physics in organic materials in spite of the weak atomic spin-orbit coupling.