Research Article
Multi-agent Deep Reinforcement Learning Based Adaptive User Association in Heterogeneous Networks
@INPROCEEDINGS{10.1007/978-3-030-06161-6_6, author={Weiwen Yi and Xing Zhang and Wenbo Wang and Jing Li}, title={Multi-agent Deep Reinforcement Learning Based Adaptive User Association in Heterogeneous Networks}, proceedings={Communications and Networking. 13th EAI International Conference, ChinaCom 2018, Chengdu, China, October 23-25, 2018, Proceedings}, proceedings_a={CHINACOM}, year={2019}, month={1}, keywords={Heterogeneous networks User association Multi-agent Deep Q-network}, doi={10.1007/978-3-030-06161-6_6} }- Weiwen Yi
Xing Zhang
Wenbo Wang
Jing Li
Year: 2019
Multi-agent Deep Reinforcement Learning Based Adaptive User Association in Heterogeneous Networks
CHINACOM
Springer
DOI: 10.1007/978-3-030-06161-6_6
Abstract
Nowadays, lots of technical challenges emerge focusing on user association in ever-increasingly complicated 5G heterogeneous networks. With distributed multiple attribute decision making (MADM) algorithm, users tend to maximize their utilities selfishly for lack of cooperation, leading to congestion. Therefore, it is efficient to apply artificial intelligence to deal with these emerging problems, which enables users to learn with incomplete environment information. In this paper, we propose an adaptive user association approach based on multi-agent deep reinforcement learning (RL), considering various user equipment types and femtocell access mechanisms. It aims to achieve a desirable trade-off between Quality of Experience (QoE) and load balancing. We formulate user association as a Markov Decision Process. And a deep RL approach, semi-distributed deep Q-network (DQN), is exploited to get the optimal strategy. Individual reward is defined as a function of transmission rate and base station load, which are adaptively balanced by a designed weight. Simulation results reveal that DQN with adaptive weight achieves the highest average reward compared with DQN with fixed weight and MADM, which indicates it obtains the best trade-off between QoE and load balancing. Compared with MADM, our approach improves by , , in terms of QoE, load balancing and blocking probability, respectively. Furthermore, semi-distributed framework reduces computational complexity.
