The origin of exciton-phonon interaction in polar semiconductors is investigated. The relative contributions of Fröhlich coupling and deformation potential scattering are identified by analyzing experimentally measured phonon-assisted luminescence using a rigorous many-body approach. Our experiment-theory comparison demonstrates that phonon scattering is significantly influenced by many-body interactions. Fröhlich interaction can be strongly suppressed for excitons even when it dominates electronic single particle scattering. The results show that deformation potential scattering dominates the exciton-phonon interaction in ZnO, whereas Fröhlich interaction prevails in CdS, and both coupling mechanisms yield almost equal contributions in ZnS.