This study introduces a novel Hierarchical Reinforcement Learning (HRL) based traffic control system, employing a two-level RL approach to optimize signal timing at urban intersections. The primary RL agent adjusts green phase durations, while the secondary agent determines transitions to flashing light mode based on intersection conditions to alleviate traffic during low-traffic periods. This system effectively integrates pedestrian and vehicular dynamics and ensures adherence to practical constraints like phase sequence and green time limitations. Comparative analysis with conventional methods shows our approach significantly reduces waiting times, vehicle stops, and fuel consumption. By using both synthetic and real-world data, our results demonstrate a robust improvement in traffic flow efficiency, offering promising implications for urban traffic management.