In a preregistered, cross-sectional study, we investigated whether olfactory loss is a reliable predictor of COVID-19 using a crowdsourced questionnaire in 23 languages to assess symptoms in individuals self-reporting recent respiratory illness. We quantified changes in chemosensory abilities during the course of the respiratory illness using 0-100 visual analog scales (VAS) for participants reporting a positive (C19+; n = 4148) or negative (C19-; n = 546) COVID-19 laboratory test outcome. Logistic regression models identified univariate and multivariate predictors of COVID-19 status and post-COVID-19 olfactory recovery. Both C19+ and C19- groups exhibited smell loss, but it was significantly larger in C19+ participants (mean ± SD, C19+: -82.5 ± 27.2 points; C19-: -59.8 ± 37.7). Smell loss during illness was the best predictor of COVID-19 in both univariate and multivariate models (ROC AUC = 0.72). Additional variables provide negligible model improvement. VAS ratings of smell loss were more predictive than binary chemosensory yes/no-questions or other cardinal symptoms (e.g., fever). Olfactory recovery within 40 days of respiratory symptom onset was reported for ~50% of participants and was best predicted by time since respiratory symptom onset. We find that quantified smell loss is the best predictor of COVID-19 amongst those with symptoms of respiratory illness. To aid clinicians and contact tracers in identifying individuals with a high likelihood of having COVID-19, we propose a novel 0-10 scale to screen for recent olfactory loss, the ODoR-19. We find that numeric ratings ≤2 indicate high odds of symptomatic COVID-19 (4 < OR < 10). Once independently validated, this tool could be deployed when viral lab tests are impractical or unavailable.