Background: The COVID-19 outbreak necessitated physical distancing, as part of secondary prevention, at a personal and professional level. Working from home (WFH) became increasingly important. In this study, the impact of the COVID-19 pandemic restrictions on physical and mental health is investigated, compared with pre-pandemic data, and with employees who WFH and are on-site. Methods: Data from the German Saxon longitudinal study population were used. Attitudes towards WFH as well as mental and physical health assessments during the COVID-19 pandemic were examined. Comparisons were made with corresponding pre-pandemic scores and between employees WFH and on-site in 2022. Results: In total, 319 participants with equal gender distribution were included. Of those, 86 worked from home stating better organizability of their work, more time for partnership, less stress, and greater work satisfaction. Compared to pre-pandemic data, the D-score, PHQ-4, G-Score, and PHQ-SSS-8 showed a significant increase. No difference in physical or mental health between employees WFH and on-site was observed. Conclusion: In general, COVID-19 restrictions had a negative impact on mental and physical health. Although WFH is well accepted, it did not show significant health benefits.