We report thermodynamic, neutron diffraction, and inelastic neutron scattering measurements on Cs₂RuO₄, a member of the celebrated family of frustrated magnets Cs₂MX₄ (M = Cu, Co, X = Br, Cl). Unlike the previously studied members, it is based on 4d transition metal ions with S = 1. Mapping out the H-T magnetic phase diagram reveals an unusual continuous spin-flop-like phase transition associated with a quantum critical point within the antiferromagnetically ordered phase. A quantitative analysis of the complex magnetic excitation spectrum measured in zero field allows us to derive a model magnetic Hamiltonian for this compound. Its main feature is a frustration of magnetic anisotropy on a level that is much higher than in any of the previously studied species. This frustration naturally explains the peculiar phase transition observed.