Common experience suggests that attracting invariant sets in nonlinear dynamical systems are generally stable. Contrary to this intuition, we present a dynamical system, a network of pulse-coupled oscillators, in which unstable attractors arise naturally. From random initial conditions, groups of synchronized oscillators (clusters) are formed that send pulses alternately, resulting in a periodic dynamics of the network. Under the influence of arbitrarily weak noise, this synchronization is followed by a desynchronization of clusters, a phenomenon induced by attractors that are unstable. Perpetual synchronization and desynchronization lead to a switching among attractors. This is explained by the geometrical fact, that these unstable attractors are surrounded by basins of attraction of other attractors, whereas the full measure of their own basin is located remote from the attractor. Unstable attractors do not only exist in these systems, but moreover dominate the dynamics for large networks and a wide range of parameters.