A physics-informed reinforcement learning-based strategy for local and coordinated ramp metering

Research output: Contribution to journalResearch articleContributedpeer-review



This paper proposes a physics-informed reinforcement learning(RL)-based ramp metering strategy, which trains the RL model using a combination of historic data and synthetic data generated from a traffic flow model. The optimal policy of the RL model is updated through an iterative training process, where in each iteration a new batch of historic data is collected and fed into the training data set. Such iterative training process can evaluate the control policy from reality rather than from a simulator, thus avoiding the RL model being trapped in an inaccurate training environment. The proposed strategy is applied to both local and coordinated ramp metering. Results from extensive microscopic simulation experiments demonstrate that the proposed strategy (i) significantly improves the traffic performance in terms of total time spent savings; (ii) outperforms classical feedback-based ramp metering strategies; and (iii) achieves higher improvements than an existing RL-based ramp metering strategy, which trains the RL model merely by a simulator. We also test the performance of two different learning algorithms in the simulation experiment, namely a conventional tabular approach and a batch-constrained deep RL approach. It is found that the deep RL approach is not as effective as the conventional tabular approach in the proposed strategy due to the limited amount of training data.


Original languageEnglish
Article number103584
JournalTransportation Research Part C: Emerging Technologies
Publication statusPublished - 1 Apr 2022

External IDs

Scopus 85124615671
unpaywall 10.1016/j.trc.2022.103584
Mendeley 4ff85f2b-3d7f-3e87-a77b-0ba5bbb59c85


ASJC Scopus subject areas


  • Data-driven approach, Ramp metering, Reinforcement learning, Traffic control