Assessing the safety of complex Cyber-Physical Systems (CPS) is a challenge in any industry. Fault Injection (FI) is a proven technique for safety analysis and is recommended by the automotive safety standard ISO 26262. Traditional FI methods require a considerable amount of effort and cost as FI is applied late in the development cycle and is driven by manual effort or random algorithms. In this paper, we propose a Reinforcement Learning (RL) approach to explore the fault space and find critical faults. During the learning process, the RL agent injects and parameterizes faults in the system to cause catastrophic behavior. The fault space is explored based on a reward function that evaluates previous simulation results such that the RL technique tries to predict improved fault timing and values. In this paper, we apply our technique on an Adaptive Cruise Controller with sensor fusion and compare the proposed method with Monte Carlo-based fault injection. The proposed technique is more efficient in terms of fault coverage and time to find the first critical fault.