The aim of this study was to investigate whether COVID-associated olfactory impairment differs from olfactory disorders due to other upper respiratory tract infections. We investigated the frequency of a SARS-CoV-2 infection among subjects presenting with a subjective olfactory impairment to a corona outpatient clinic between October 2020 and March 2021. Olfactory and gustatory loss were tested psychophysically, and the type of infection, SARS-CoV-2 versus 14 other common cold viruses, was assessed with nasopharyngeal swabs. Differences between the smell impairment caused by the pathogens were compared. Out of the 2120 patients, 314 reported sudden smell and/or taste loss (14%). In 68.9% of them, olfactory and in 25.6%, gustatory dysfunction could be confirmed by psychophysical testing. Of those with a psychophysically determined loss of smell, 61% were tested positive for SARS-CoV-2. SARS-CoV-2 led to a significantly more severe loss of smell and more qualitative olfactory disorders than other pathogens. Apart from rhinorrhea, shortness of breath and sore throat accompanying cold symptoms do not differ significantly between the viruses indicating the particular importance of smell loss in the differential diagnosis of seasonal colds. Multiplex-PCR in non-COVID patients revealed that only 27% of them had rhinoviruses, whereas the remainder were no further identified pathogens. Olfactory screening significantly increases diagnostic accuracy in COVID-19 patients compared to subjective assessment of olfactory loss.