We report the temperature-dependent magnetic and structural properties of epitaxial Mn5Ge3 thin films grown on Ge substrates. Utilizing density-functional theory (DFT) calculations and various experimental methods, we reveal mechanisms governing the switching between collinear and noncollinear spin configuration in Mn5Ge3. The Mn atoms in Mn5Ge3 occupy two distinct Wyckoff positions with fourfold (Mn-1) and sixfold (Mn-2) multiplicity. The DFT calculations reveal that below a critical distance of approximately 3.002 angstrom the coupling between Mn-2 atoms is antiferromagnetic (AFM) while ferromagnetic (FM) above that critical distance. The FM coupling between Mn-1 atoms is weakly affected by the strain. The observed noncollinear spin configuration is due to the coexistence of AFM and FM coupling at low temperatures. The findings give insight in developing strain-controlled spintronic devices.