4th International ICST Conference on Cognitive Radio Oriented Wireless Networks and Communications

Research Article

Reinforcement learning based spectrum-aware routing in multi-hop cognitive radio networks

  • @INPROCEEDINGS{10.1109/CROWNCOM.2009.5189189,
        author={Bing  Xia and Muhammad Husni  Wahab and Yang  Yang and Zhong Fan and Mahesh Sooriyabandara},
        title={Reinforcement learning based spectrum-aware routing in multi-hop cognitive radio networks},
        proceedings={4th International ICST Conference on Cognitive Radio Oriented Wireless Networks and Communications},
        publisher={IEEE},
        proceedings_a={CROWNCOM},
        year={2009},
        month={8},
        keywords={Cognitive radio network; routing; reinforcement  learning},
        doi={10.1109/CROWNCOM.2009.5189189}
    }
    
  • Bing Xia
    Muhammad Husni Wahab
    Yang Yang
    Zhong Fan
    Mahesh Sooriyabandara
    Year: 2009
    Reinforcement learning based spectrum-aware routing in multi-hop cognitive radio networks
    CROWNCOM
    IEEE
    DOI: 10.1109/CROWNCOM.2009.5189189
Bing Xia1,2,*, Muhammad Husni Wahab1,2,*, Yang Yang1,2,*, Zhong Fan3,4,*, Mahesh Sooriyabandara3,4,*
  • 1: Department of Electronic and Electrical Engineering, University College London,
  • 2: London, United Kingdom
  • 3: Telecommunications Research Laboratory, Toshiba Research Europe Ltd.
  • 4: Bristol, United Kingdom
*Contact email: bxia@ee.ucl.ac.uk, m.wahab@ee.ucl.ac.uk, y.yang@ee.ucl.ac.uk, zhong.fan@toshiba-trel.com, mahesh.sooriyabandara@toshiba-trel.com

Abstract

Routing in multi-hop cognitive radio networks (CRN) should be spectrum-aware. In this paper, two adaptive reinforcement learning based spectrum-aware routing protocols are introduced. Q-learning and dual reinforcement learning are applied respectively for them. The cognitive nodes store a table of Q values that estimate the numbers of available channels on the routes and update them while routing. So they can adaptively learn good routes which have more available channels from just local information. Compared to the previous spectrum aware routing protocols in multi-hop cognitive radio networks, they are simpler and easier to implement, more cost-effective, and can avoid drawbacks in on-demand protocols but still keep adaptive and dynamic routing. Both of our protocols perform better than the spectrum-aware shortest path protocol when network load is not too low. In the meantime, spectrum-aware DRQ-routing learns the optimal routing policy 1.5 times as fast as the spectrum-aware Q-routing at low and medium network load. It also learns a routing policy which is more than seven times as good as that of spectrum-aware Q-routing at high network load.