Altered biophysical properties of cancer cells and of their microenvironment contribute to cancer progression. While the relationship between microenvironmental stiffness and cancer cell mechanical properties and responses has been previously studied using two-dimensional (2D) systems, much less is known about it in a physiologically more relevant 3D context and in particular for multicellular systems. To investigate the influence of microenvironment stiffness on tumor spheroid mechanics, we first generated MCF-7 tumor spheroids within matrix metalloproteinase (MMP)-degradable 3D polyethylene glycol (PEG)-heparin hydrogels, where spheroids showed reduced growth in stiffer hydrogels. We then quantitatively mapped the mechanical properties of tumor spheroids in situ using Brillouin microscopy. Maps acquired for tumor spheroids grown within stiff hydrogels showed elevated Brillouin frequency shifts (hence increased longitudinal elastic moduli) with increasing hydrogel stiffness. Maps furthermore revealed spatial variations of the mechanical properties across the spheroids’ cross-sections. When hydrogel degradability was blocked, comparable Brillouin frequency shifts of the MCF-7 spheroids were found in both compliant and stiff hydrogels, along with similar levels of growth-induced compressive stress. Under low compressive stress, single cells or free multicellular aggregates showed consistently lower Brillouin frequency shifts compared to spheroids growing within hydrogels. Thus, the spheroids’ mechanical properties were modulated by matrix stiffness and degradability as well as multicellularity, and also to the associated level of compressive stress felt by tumor spheroids. Spheroids generated from a panel of invasive breast, prostate and pancreatic cancer cell lines within degradable stiff hydrogels, showed higher Brillouin frequency shifts and less cell invasion compared to those in compliant hydrogels. Taken together, our findings contribute to a better understanding of the interplay between cancer cells and microenvironment mechanics and degradability, which is relevant to better understand cancer progression.