Multi-agent Reinforcement Learning for Cooperative On-Ramp Merging of Connected Automated Vehicles

Ramp merging areas on highways often serve as bottleneck areas, leading to frequent interactions and accidents between vehicles on the ramp and the arterial road. This results in severe congestion and reduced traffic performance. The emergence of Connected Autonomous Vehicles (CAVs) offers advanced solutions to address these issues and improve traffic operations at ramp merging areas. While previous studies have explored CAV decision-making approaches such as optimization control, model predictive control, and reinforcement learning, they face difficulties in accurately modeling the complex and dynamic scenarios of ramp merging. To overcome these challenges, this paper proposes a collaborative decision-making and control model based on Multi-agent Reinforcement Learning (MARL) for mixed vehicles (CAV-HDV) in multi-lane ramp merging scenarios on arterial roads. The paper introduces three novel MARL algorithms and conducts simulations in six different scenarios to evaluate traffic performance under various lane numbers and traffic densities. The results demonstrate the effectiveness of the proposed collaborative model for ramp merging vehicles. The proposed algorithms significantly reduce collision rates and improve traffic efficiency.