Objective: In this contribution, we present an automated sleep stage classifier using modern machine learning methods with cardiorespiratory time series as input signals. We investigate (1) which classification quality we can yield, (2) how variations in the input data affect the results and (3) to which extend the proposed classifier model can serve as basis for medical diagnoses. Methods: We extracted R-to-R intervals from electrocardiograms and breath-to-breath intervals from thoracic respiratory effort. Our analysis used 5036 patients from the Sleep Heart Health Study, 998 random patients served as independent hold-out test data. Our model architecture consists of convolutional and recurrent layers. From the predicted hypnograms, we calculated sleep metrics to estimate if the proposed model can serve as basis for medical diagnoses. We further modified the RR interval data and trained new models on that data to investigate its robustness to different data sources and resulting classification quality. Results: The proposed method yields a Cohen's kappa score of 0.80 for distinguishing Wakefulness, NREM and REM, which is considered an almost perfect classification. Further, the proposed method is very robust concerning differing data sources. We found that some summarizing sleep metrics calculated from our predictions are reliable (e.g. sleep efficiency), others, however, are rather unreliable (e.g. percentage of deep sleep). Conclusions: Our results demonstrate the applicability and robustness of the proposed method for sleep staging. Additionally, they underline the requirement to consider not only scoring accuracy but also sleep metrics.