The design of reinforcement learning (RL) agents is difficult, especially in domains with complex and possibly conflicting objectives such as autonomous driving. In addition to the formal nature of RL with high technical barriers, the fragility of the reward signal results in the common trial-and-error practice in the design of RL agents. We propose a novel goal-oriented specification language that is tailored to reinforcement learning but abstracts from technical details. To overcome the problematic trial-and-error practice, our specification language provides the foundation for an easy and systematic design process in RL.