Quantifying soil carbon dynamics is of utmost relevance in the context of global change because soils play an important role in land–atmosphere gas exchange. Our current understanding of both present and future carbon dynamics is limited because we fail to accurately represent soil processes across temporal and spatial scales, partly because of the paucity of data on the relative importance and hierarchical relationships between microbial, geochemical and climatic controls. Here, using observations from a 3,000-kyr-old soil chronosequence preserved in alluvial terrace deposits of the Merced River, California, we show how soil carbon dynamics are driven by the relationship between short-term biotic responses and long-term mineral weathering. We link temperature sensitivity of heterotrophic respiration to biogeochemical soil properties through their relationship with microbial activity and community composition. We found that soil mineralogy, and in particular changes in mineral reactivity and resulting nutrient availability, impacts the response of heterotrophic soil respiration to warming by altering carbon inputs, carbon stabilization, microbial community composition and extracellular enzyme activity. We demonstrate that biogeochemical alteration of the soil matrix (and not short-term warming) controls the composition of microbial communities and strategies to metabolize nutrients. More specifically, weathering first increases and then reduces nutrient availability and retention, as well as the potential of soils to stabilize carbon.