Grain growth in polycrystals is traditionally considered a capillarity-driven process, where grain boundaries (GBs) migrate toward their centers of curvature (i.e., mean curvature flow) with a velocity proportional to the local curvature (including extensions to account for anisotropic GB energy and mobility). Experimental and simulation evidence shows that this simplistic view is untrue. We demonstrate that the failure of the classical mean curvature flow description of grain growth mainly originates from the shear deformation naturally coupled with GB motion (i.e., shear coupling). Our findings are built on large-scale microstructure evolution simulations incorporating the fundamental (crystallography-respecting) microscopic mechanism of GB migration. The nature of the deviations from curvature flow revealed in our simulations is consistent with observations in recent experimental studies on different materials. This work also demonstrates how to incorporate the mechanical effects that are essential to the accurate prediction of microstructure evolution.