The competition among superconductivity, stripe-type magnetic order, and a new type of C4 symmetric magnetic order in Ba1-xKxFe2As2 is theoretically studied, focusing on its impact on the global phase diagram. By carrying out a renormalization group analysis of an effective field theory, we obtain the energy-scale dependent flows of all the model parameters, and then apply the results to understand the observed phase diagram. On the basis of the renormalization group analysis, we show that the critical line of nematic order has a negative slope in the superconducting dome and superconductivity is suppressed near the magnetic quantum critical point, which are both consistent with recent experiments. Moreover, we find that, although the observed C4 symmetric magnetic state could be a charge-spin density wave or a spin-vortex crystal at high temperatures, charge-spin density wave is the only stable C4 magnetic state in the low-temperature regime. Therefore, ordering competition provides a method to distinguish these two candidate C4 magnetic states.