Decentralized Cognitive MAC Protocol Design Based on POMDP and Q-Learning

Zhongli Lan; Hong Jiang; Xiaoli Wu

7th International Conference on Communications and Networking in China

Research Article

Decentralized Cognitive MAC Protocol Design Based on POMDP and Q-Learning

Cite: BibTeX Plain Text

@INPROCEEDINGS{10.1109/ChinaCom.2012.6417543,
    author={Zhongli Lan and Hong Jiang and Xiaoli Wu},
    title={Decentralized Cognitive MAC Protocol Design Based on POMDP and Q-Learning},
    proceedings={7th International Conference on Communications and Networking in China},
    publisher={IEEE},
    proceedings_a={CHINACOM},
    year={2012},
    month={10},
    keywords={q-learning pomdp cognitive mac protocol decentralized},
    doi={10.1109/ChinaCom.2012.6417543}
}

Zhongli Lan
Hong Jiang
Xiaoli Wu
Year: 2012
Decentralized Cognitive MAC Protocol Design Based on POMDP and Q-Learning
CHINACOM
IEEE
DOI: 10.1109/ChinaCom.2012.6417543

Zhongli Lan¹^,*, Hong Jiang¹, Xiaoli Wu²

1: Southwest University of Science and Technology
2: China Academy of Engineering Physics

*Contact email: albert.z.lan@gmail.com

Abstract

A decentralized cognitive MAC protocol is proposed in this paper, whose core depends on the partially observed Markov process (POMDP) and Q-Learning. Limited by the hardware and environment, a secondary user may not be able to have ability to sense the entire spectrum space. Therefore, the POMDP is exploited to model the secondary network. In this paper, Q-Learning is applied to solve the POMDP because it can make full advantage of the past observation and decision experiences to optimize current action, and needs not transfer the POMDP into a belief MDP. The numeral simulation results show that Q-Learning-based decentralized cognitive MAC protocol improves the overall performance of the networks.

Keywords: q-learning pomdp cognitive mac protocol decentralized

Published: 2012-10-25
Publisher: IEEE

: http://dx.doi.org/10.1109/ChinaCom.2012.6417543