We present and analyze the first example of a dynamical system that naturally exhibits attracting periodic orbits that are unstable. These unstable attractors occur in networks of pulse-coupled oscillators, and become prevalent with increasing network size for a wide range of parameters. They are enclosed by basins of attraction of other attractors but are remote from their own basin volume such that arbitrarily small noise leads to a switching among attractors.