Extreme climate events constitute a major risk to global food production. Among these, extreme rainfall is often dismissed from historical analyses and future projections, the impacts and mechanisms of which remain poorly understood. Here we used long-term nationwide observations and multi-level rainfall manipulative experiments to explore the magnitude and mechanisms of extreme rainfall impacts on rice yield in China. We find that rice yield reductions due to extreme rainfall were comparable to those induced by extreme heat over the last two decades, reaching 7.6 ± 0.9% (one standard error) according to nationwide observations and 8.1 ± 1.1% according to the crop model incorporating the mechanisms revealed from manipulative experiments. Extreme rainfall reduces rice yield mainly by limiting nitrogen availability for tillering that lowers per-area effective panicles and by exerting physical disturbance on pollination that declines per-panicle filled grains. Considering these mechanisms, we projected ~8% additional yield reduction due to extreme rainfall under warmer climate by the end of the century. These findings demonstrate that it is critical to account for extreme rainfall in food security assessments.