Background: Studies show that three-quarters of mental disorders appear during young adulthood, which makes students a risk group. Especially people with anxiety and depression experience lower Quality of Life (QoL) compared to healthy persons. Furthermore, previous research found that there was a wide range of negative mental consequences triggered by the COVID-19 pandemic. This study aimed to examine the association between anxiety, depression and QoL in male and female students at the time of the COVID-19 pandemic. Methods: 297 German students (121 men, age span_men: 18–41 years; 176 women, age span_women: 18–52 years) filled in the following questionnaires: World Health Organization Quality of Life Brief Version, Hospital Anxiety and Depression Scale and the Symptom-Checklist-90-R. Men and women did not differ significantly in their physical, psychological, environmental and global QoL. Results: While women showed higher raw anxiety scores, groups did not differ in terms of their raw depression scores. Furthermore, we found main effects of anxiety and depression on the four QoL subscales. Students´ QoL was highest if they were not affected by anxiety and depression, independently of gender. Psychological and social QoL was worst if the students reported marginal and particularly clinically significant levels of anxiety and depression. Men experienced worse psychological and social QoL than women for clinically significant anxiety levels. Conclusions: Interventions should target especially the psychological and the social subscales of QoL, as these areas are most affected by anxiety and depression. Possible interventions could be psychoeducational programs or participation in sports because it offers an opportunity for social interaction and goal-directed activity.