Increasing evidence indicates that the heterogeneous tumor stroma supports therapy resistance at multiple levels. Fibroblasts, particularly cancer-associated fibroblasts (CAFs) are critical components of the tumor stroma. However, the impact of CAFs on the outcome of radiotherapy (RT) is poorly understood. Here, we investigated if and how fibroblasts/CAFs modulate the radiation response of malignant tumors by altering cancer cell radiosensitivity or radioresistance in vitro and in vivo. The influence of fibroblasts on cancer cell proliferation, cell death induction and long-term survival after RT was studied using different sets of fibroblasts and cancer cells in an indirect co-culture (2D) system to analyse potential paracrine interactions or a 3D model to study direct interactions. Paracrine signals from embryonic NIH-3T3 fibroblasts promoted MPR31.4 prostate and Py8119 breast cancer cell proliferation. Indirect co-culture with L929 skin fibroblasts induced higher levels of apoptosis in irradiated MPR31.4 cells, while they promoted proliferation of irradiated Py8119 cells. In addition, NIH-3T3 fibroblasts promoted long-term clonogenic survival of both tumor cell types upon irradiation in the 3D co-culture system when compared to non-irradiated controls. Also in vivo, co-implantation of cancer cells and fibroblasts resulted in different effects depending on the respective cell combinations used: co-implantation of MPR31.4 cells with NIH-3T3 fibroblasts or of Py8119 cells with L929 fibroblasts led to increased tumor growth and reduced radiation-induced tumor growth delay when compared to the respective tumors without co-implanted fibroblasts. Taken together, the impact of fibroblasts on cancer cell behavior and radiation sensitivity largely depended on the respective cell types used as they either exerted a pro-tumorigenic and radioresistance-promoting effect, an anti-tumorigenic effect, or no effect. We conclude that the plasticity of fibroblasts allows for such a broad spectrum of activities by the same fibroblast and that this plasticity is at least in part mediated by cancer cell-induced fibroblast activation toward CAFs.