There is increasing interest in automating train operations of mainline services, e.g., to increase network capacity. Automatic train operation (ATO) is already achieved by several pilot projects, but is still not implemented on a large scale. Functional, interoperability and performance tests are necessary before ATO can be introduced generally. Virtual preliminary analysis will contribute to the validation process to ensure a safe and successful implementation. This paper aims to present an approach that applies to the performance testing of ATO systems. Therefore, methods and test standards for technologies enabling automatic operation in other transport sectors are reviewed. The main findings have been adapted, transformed and combined to be used as a general strategy for virtual performance testing in the railway sector. Specifically, universal performance indicators commonly used in the railway sector, namely punctuality, accuracy, energy consumption, safety and comfort, are presented. They are refined by adding sub-indicators specific to the performance evaluation of ATO algorithms. A layer model for scenario description is adapted from the automotive sector, as well as the definition of different scenario types. Lastly, factors that can influence the performance of an ATO algorithm are identified. For demonstration purposes, a simple case study is conducted. Thereby we exemplarily show-cased the approach for ATO performance testing using a microscopic train simulator in combination with an ATO algorithm.