Using resonant x-ray diffraction, we observe an easy c-axis collinear antiferromagnetic structure for the bilayer Sr ₃Ir ₂O ₇, a significant contrast to the single layer Sr ₂IrO ₄ with in-plane canted moments. Based on a microscopic model Hamiltonian, we show that the observed spin-flop transition as a function of number of IrO ₂ layers is due to strong competition among intra- and interlayer bond-directional pseudodipolar interactions of the spin-orbit entangled J _eff=1/2 moments. With this we unravel the origin of anisotropic exchange interactions in a Mott insulator in the strong spin-orbit coupling regime, which holds the key to the various types of unconventional magnetism proposed in 5d transition metal oxides.