In an ongoing effort towards a more sustainable rare-earth element market, there is a high potential for an efficient recycling of rare-earth elements from end-of-life compact fluorescent lamps by physical separation of the individual phosphors. In this study, we investigate the separation of five fluorescent lamp particles by high-gradient magnetic separation in a rotary permanent magnet separator. We thoroughly characterize the phosphors by ICP-MS, laser diffraction analysis, gas displacement pycnometry, surface area analysis, SQUID-VSM, and Time-Resolved Laser-Induced Fluorescence Spectroscopy. We present a fast and reliable quantification method for mixtures of the investigated phosphors, based on a combination of Time-Resolved Laser-Induced Fluorescence Spectroscopy and parallel factor analysis. With this method, we were able to monitor each phosphors’ removal dynamics in the high-gradient magnetic separator and we estimate that the particles’ removal efficiencies are proportional to (d^2·χ)1/3 . Finally, we have found that the removed phosphors can readily be recovered easily from the separation cell by backwashing with an intermittent air–water flow. This work should contribute to a better understanding of the phosphors’ separability by high-gradient magnetic separation and can simultaneously be considered to be an important preparation for an upscalable separation process with (bio)functionalized superparamagnetic carriers.