Research Article
Accelerating Spectrum Sharing Algorithms for Cognitive Radio Transmitters in a Momentum Q-Learning Approach
@INPROCEEDINGS{10.1007/978-3-030-72792-5_42, author={Lianghui Zhu and Zhanke Zhou and Zhaochuan Peng and Xiaojun Hei}, title={Accelerating Spectrum Sharing Algorithms for Cognitive Radio Transmitters in a Momentum Q-Learning Approach}, proceedings={Simulation Tools and Techniques. 12th EAI International Conference, SIMUtools 2020, Guiyang, China, August 28-29, 2020, Proceedings, Part I}, proceedings_a={SIMUTOOLS}, year={2021}, month={4}, keywords={Reinforcement learning Cognitive radio Spectrum sharing}, doi={10.1007/978-3-030-72792-5_42} }- Lianghui Zhu
Zhanke Zhou
Zhaochuan Peng
Xiaojun Hei
Year: 2021
Accelerating Spectrum Sharing Algorithms for Cognitive Radio Transmitters in a Momentum Q-Learning Approach
SIMUTOOLS
Springer
DOI: 10.1007/978-3-030-72792-5_42
Abstract
The radio frequency spectrum is a scarce resource and cognitive radio has been under heavy research to improve the utilization of spectrum in the past thirty years. It is crucial to optimize the performance of cognitive radio for high values for practical applications while it has turned out to be very technically challenging. The conventional cognitive radio methods have strong pertinence and coupling because they are generally designed for a specific application environment. To address the problem of spectrum sharing with collision avoidance mechanisms in cognitive radio, in this paper we propose a new momentum-based Q-learning algorithm to accelerate reinforcement learning based spectrum sharing algorithms for cognitive radio transmitters. We conduct a performance evaluation study based on a simulation toolkit for the reinforcement learning research and the ns-3 network simulator “ns3-gym”. As a demonstrating case study, the proposed algorithm is able to capture the learnable patterns from a periodic channel occupation in a wireless environment and avoid channel collision effectively, finally improving channel efficiency and reducing the end-to-end time delay. The simulation results demonstrated that our proposed momentum Q-learning algorithm achieves a lower collision rate, faster convergence as well as stronger generalization capacity compared with two conventional algorithms including a greedy algorithm and a deep Q-learning network algorithm.
