Reinforcement Learning (RL) might be very promising for solving a variety of challenges in the field of autonomous driving due to its ability to find long-term oriented solutions in complex decision scenarios. For training and validation of a RL algorithm, a simulative environment is advantageous due to risk reduction and saving of resources. This contribution presents an RL environment designed for the optimization of longitudinal control. The focus is on providing an illustrative and comprehensible example for a continuous real-world problem. The environment will be published following the OpenAI Gym interface, allowing for easy testing and comparing of novel RL algorithms. In addition to details on implementation reference is also made to areas where research is required.