We study magnetic order induced by non-magnetic impurities in quantum paramagnets with incommensurate host spin correlations. In contrast to the well-studied commensurate case where the defect-induced magnetism is spatially disordered but non-frustrated, the present problem combines strong disorder with frustration and, consequently, leads to spin-glass order. We discuss the crossover from strong randomness in the dilute limit to more conventional glass behavior at larger doping, and numerically characterize the robust short-range order inherent to the spin-glass phase. We relate our findings to magnetic order in both BiCu ₂PO ₆ and YBa ₂Cu ₃O _6.6 induced by Zn substitution.