Storm waves transport and sort coarse gravel along coasts. This fundamental process is important under changing sea-levels and increased storm frequency and intensity. However, limited information on intra-storm clast motion restricts theory development for coastal gravel sorting and coastal management of longshore transport. Here, we use smart boulders equipped with loggers recording underwater, real-time, intra-storm clast motion, and measured longshore displacement of varied-mass marked boulders during storms. We utilize the unique setting of the Dead Sea shores where rapidly falling water levels allow isolating boulder transport during individual storms. Guided by these observations, we develop a new model quantifying the critical wave height for a certain clast mass mobilization. Then, we obtain an expression for the longshore clast displacement under the fluid-induced pressure impulse of a given wave. Finally, we formulate the sorting enforced by wave-height distributions during a storm, demonstrating how sorting is a direct manifestation of regional hydroclimatology.