Communications and Networking. 13th EAI International Conference, ChinaCom 2018, Chengdu, China, October 23-25, 2018, Proceedings

Research Article

Multi-agent Deep Reinforcement Learning Based Adaptive User Association in Heterogeneous Networks

Download
182 downloads
  • @INPROCEEDINGS{10.1007/978-3-030-06161-6_6,
        author={Weiwen Yi and Xing Zhang and Wenbo Wang and Jing Li},
        title={Multi-agent Deep Reinforcement Learning Based Adaptive User Association in Heterogeneous Networks},
        proceedings={Communications and Networking. 13th EAI International Conference, ChinaCom 2018, Chengdu, China, October 23-25, 2018, Proceedings},
        proceedings_a={CHINACOM},
        year={2019},
        month={1},
        keywords={Heterogeneous networks User association Multi-agent Deep Q-network},
        doi={10.1007/978-3-030-06161-6_6}
    }
    
  • Weiwen Yi
    Xing Zhang
    Wenbo Wang
    Jing Li
    Year: 2019
    Multi-agent Deep Reinforcement Learning Based Adaptive User Association in Heterogeneous Networks
    CHINACOM
    Springer
    DOI: 10.1007/978-3-030-06161-6_6
Weiwen Yi1,*, Xing Zhang1, Wenbo Wang1, Jing Li1
  • 1: Beijing University of Posts and Telecommunications
*Contact email: yww2013@bupt.edu.cn

Abstract

Nowadays, lots of technical challenges emerge focusing on user association in ever-increasingly complicated 5G heterogeneous networks. With distributed multiple attribute decision making (MADM) algorithm, users tend to maximize their utilities selfishly for lack of cooperation, leading to congestion. Therefore, it is efficient to apply artificial intelligence to deal with these emerging problems, which enables users to learn with incomplete environment information. In this paper, we propose an adaptive user association approach based on multi-agent deep reinforcement learning (RL), considering various user equipment types and femtocell access mechanisms. It aims to achieve a desirable trade-off between Quality of Experience (QoE) and load balancing. We formulate user association as a Markov Decision Process. And a deep RL approach, semi-distributed deep Q-network (DQN), is exploited to get the optimal strategy. Individual reward is defined as a function of transmission rate and base station load, which are adaptively balanced by a designed weight. Simulation results reveal that DQN with adaptive weight achieves the highest average reward compared with DQN with fixed weight and MADM, which indicates it obtains the best trade-off between QoE and load balancing. Compared with MADM, our approach improves by , , in terms of QoE, load balancing and blocking probability, respectively. Furthermore, semi-distributed framework reduces computational complexity.