Background: Virtual Reality (VR) does not only include the use of stereoscopic images, but also possibilities for an interaction with and participation in a computer-generated environment. However, laboratory studies primarily focus on the first part of the definition only. In this context, comparing results from different VR applications with diverging goals becomes difficult. This is especially true in the field of cybersickness research (visually induced motion sickness in VR), as self-report symptom questionnaires are used. The prominent Simulator Sickness Questionnaire (SSQ) is criticized for the lack of specificity, the double factorial loadings, the outdatedness, and the unrepresentative sample. VR-specific revisions like the Virtual Reality Sickness Questionnaire (VRSQ) address these criticisms but lack generalizability. Methods: The current paper uses a Confirmatory Factor Analysis of the VRSQ with data from three different VR environments and a sample size of N = 244. The environments had different setups, visual complexities, and interaction possibilities. These characteristics influenced the factorial structure of the VRSQ as a moderator. Furthermore, to control for VR-unrelated effects Baseline ratings were taken into account. Results: The Confirmatory Factor Analysis indicated a moderate fit for the global model, but a misspecification for two of the three environments. Only the environment similar to the original VRSQ paper converged with the model. Conclusions: In conclusion, a detailed description of the VR environment is required in scientific method reports. Focusing on VR accessibility for physically impaired in addition to healthy subjects, an added Baseline measurement can address the discriminant validity. Until generalizable VR-specific revisions of the SSQ are validated, the paper suggests using the Δ-SSQ in aggregated raw format.