The response of vegetation physiology to drought at large spatial scales is poorly understood due to a lack of direct observations. Here, we study vegetation drought responses related to photosynthesis, evaporation, and vegetation water content using remotely sensed data, and we isolate physiological responses using a machine learning technique. We find that vegetation functional decreases are largely driven by the downregulation of vegetation physiology such as stomatal conductance and light use efficiency, with the strongest downregulation in water-limited regions. Vegetation physiological decreases in wet regions also result in a discrepancy between functional and structural changes under severe drought. We find similar patterns of physiological drought response using simulations from a soil–plant–atmosphere continuum model coupled with a radiative transfer model. Observation-derived vegetation physiological responses to drought across space are mainly controlled by aridity and additionally modulated by abnormal hydro-meteorological conditions and vegetation types. Hence, isolating and quantifying vegetation physiological responses to drought enables a better understanding of ecosystem biogeochemical and biophysical feedback in modulating climate change.