Let G be a group. A subset S of G is said to normally generate G if G is the normal closure of S in G. In this case, any element of G can be written as a product of conjugates of elements of S and their inverses. If g ∈ G and S is a normally generating subset of G, then we write ‖g‖_S for the length of a shortest word in Conj_G (S^±1):= {h⁻¹ sh|h ∈ G, s ∈ S or s⁻¹ ∈ S } needed to express g. For any normally generating subset S of G, we write ‖G‖_S = sup{‖g‖_S | g ∈ G}. Moreover, we write ∆(G) for the supremum of all ‖G‖_S, where S is a finite normally generating subset of G, and we call ∆(G) the conjugacy diameter of G. In this paper, we derive the conjugacy diameters of the semidihedral 2-groups, the generalized quaternion groups and the modular p-groups. This is a natural step after the determination of the conjugacy diameters of dihedral groups.