We report on a nonlinear scattering effect that challenges the notion of topological protection for wave packets propagating in chiral edge modes. Specifically, in a Floquet topological system close to resonant driving and with a nonlinear potential, we demonstrate how a wave packet propagating in a chiral edge mode may be irreversibly deflected by scattering off a localized wave packet, or pass the collision region virtually unaffected in an approximately linear fashion. An experimentally accessible knob to tune between those two scenarios is provided by the relative phase between the involved wave packets. This genuinely nonlinear interference phenomenon is in stark contrast to linear scattering off a static impurity, which cannot destroy a topological edge state. Besides corroborating our findings with numerically exact simulations, we propose two physical platforms where our predictions may be verified with state-of-the-art experimental techniques: first, a coupled waveguide setting where nonlinearity has been engineered via an intensity-dependent optical index, and second, a Bose-Einstein condensate of cold atoms in an optical honeycomb lattice governed by a nonlinear Gross-Pitaevskii equation that effectively accounts for many-body interactions.