Fog-Based Data Offloading in UWSNs with Discounted Rewards: A Contextual Bandit

Urban wireless sensor networks (UWSNs) are an important application scenario for the Internet of Things (IoT). Nevertheless, applications based on urban environments are often computationally intensive, and sensor nodes are resource-constrained and heterogeneous. Fog computing has the potential to liberate the computation-intensive mobile nodes through data offloading. Therefore, reliable data collection and scalable coordination based on fog computing are seen as a challenge. In this paper, the challenge of data offloading is modeled as a contextual bandit problem—an important extension of the multi-armed bandit. First, the heterogeneity of the sensor nodes is used as contextual information, allowing the network to complete data collection at a small computational cost. Second, an ever-changing environmental scenario is considered in which the distribution of re-wards is not fixed, but varies over time. Based on this non-stationary bandit model, we propose a contextual bandit algorithm NCB-rDO in order to improve the success rate of data offloading, which solves the problem of data loss when the contextual information changes suddenly. Experimental results demonstrate the effectiveness and robustness of this data offloading algorithm.