Link Scheduling for Wireless Mesh Networks Considering Gateway Feature

Based on different objectives, a variety of mathematical models for the wireless mesh network (WMNs) exist. Among them, the link scheduling model for WMNs aims at finding a data transmission schedule based on network links so that some objectives on packet transmission are optimized under certain transmission constraints. In this paper, a new WMN link scheduling model with additional information of node locations and gateway designs is orchestrated. An approximate dynamic programming algorithm is utilized for scheduling the new model. Experiment results show that, in addition to maintaining many wireless network characteristics, the proposed scheduling algorithm effectively simulates the result of dynamic programming, and has performances superior to genetic algorithm.


Introduction
The technology of wireless mesh network (WMN) has lots of advances, e.g., its specification in IEEE 802.11s has been released recently. It is well-known that the IEEE 802.11 networks have been studied broadly, e.g., IEEE 802.11 DCF priority access scheme [5], IEEE 802.11 DCF access control with optimal contention window [7], QoS issue in IEEE 802.11 WLANs [6], [8], and so on. With the development of WMNs, how maximum data packets may be transmitted in a given time to satisfy client requirements is related to the routing of network nodes and scheduling of paths between nodes. Among the various WMN model, the link scheduling model programs the links that transmit data packets between nodes to achieve certain objectives. The works in [10], [16], [20] divided time and multiplexed by routers in the WMNs model. The work in [9] described the features of WMN signals, and it may be known that the mutually interfering transmission signals between router nodes play an important role in the scheduling model. In addition, the works in [11], [19] discussed the data capacity of the router, and such nodes that may receive and transmit data packets are known to have packing constraints. Under the constraints of the programming time and node transmission mentioned above, many scheduling models with varying emphasized features have constructed their basic concepts and developed various objectives. Among these, the most common objective is to maximize data packet transmission [11], [17], [18]. Such models perform scheduling by some novel algorithms, such as dynamic programming. However, they usually did not include any gateway design and only focused on the reduction of data packet packing at nodes. Furthermore, the design of programming information placed on the links differs from actual situations in these models. Different from the setting in [13] that considered WMN consisting of only mesh routers and mesh clients, a few WMN models have considered gateway designs [1], [12], and achieved the objective of maximizing the data packets received by the gateway by scheduling with genetic algorithm. Although fair performance is obtained, their nodes do not have the function of receiving randomly arriving data packets.
Among the numerous literature of various link models, the scheduling algorithms used by designers are not quite the same, and performances to be evaluated and objectives to be achieved are also different, which includes genetic algorithm [12], dynamic programming using optimization principles, and other heuristic algorithms [17]. It is worthy to note that, among these heuristic algorithms, approximate dynamic programming (ADP) has saved much more time as compared with dynamic programming and achieved better performances as compared with other heuristic algorithms by predicting the calculation results of dynamic programming by simulation. In summary, the design of a new model conforming to features of actual WMNs and allowing different algorithms originally used in different scheduling models to compare performances is worthy of discussion, and the research emphasizes on the effect of the added features on the execution and comparison of each algorithm, the simulation effect of approximate dynamic programming, and whether the algorithm maintains the scheduling performances after expanding the extent of the model.
In this work, we have explored various different WSNs and discovered their distinct highlighted features such as the link transmission direction, node coordinates design, and various constraints; the model objectives also often differ, such as focusing on reducing the node data packing, focusing on maximizing the received packets by the gateway, or maximizing the open times of transmission links between nodes. Due to the variety of types, a model of a single type may lack certain characteristics. This work seeks to include as many WSN features as possible in the new model, such as the time slot concept of dividing programming time, the numerous constraints and randomly distributed node coordinates encountered during link transmission, and the design of the gateway location scarcely considered in conventional models. The new model adds gateway designs in the objective and achieves the objective of transmitting large amounts of packets to the gateway by maximizing the node transmission packet amount. In the discussion of the scheduling algorithm, the algorithms used for scheduling the same wireless network model are examined to ensure that the algorithms are compared under the same performance index and model designs. After selecting a superior algorithm, it is used for the comparison of scheduling performances of the designed WSN model.
Note that although the work in [14] has jointly studied routing, scheduling, and variable-width channel allocation in WMNs, their model is much different from ours.
The following sections are arranged as follows: Section 2 introduces the WSN link scheduling model and the genetic algorithm, the concept of conventional dynamic programming and approximate dynamic programming; Section 3 presents the new model and explains how the conventional dynamic programming and approximate dynamic programming are used for solutions; Section 4 gives the experiment results and analysis; Section 5 is the conclusion.

Related Work
Wireless mesh network link models proposed in the literature are discussed in this section.

Scheduling model of Spatial Time Division Multiple Access
A network structure usually is constructed by a group of sets of nodes that may connect to each other, where the nodes communicate by transmission links to transmit data packets. Long-term cautious programming is required to program the link behavior to achieve the objective of configuring the network, and thus it is not surprising that numerous research papers related to such topics have been published [3], [4], [12]. Differences in optimization objectives exist among such researches due to different network node data transmission characteristics and topology structure; however, common features of the various topics may still be deduced. In addition to solving issues of data packet transmission between nodes and their neighboring nodes, the distribution of transmission of data packets also needs attention; since each link has a fixed packing amount of data transmission and only a fixed maximum amount of data packets may be transmitted in a fixed time, the packing of other links have to be decreased to increase the packing of a certain transmission link which is anticipated to result in the reduction in the overall performances [11], [19]. Therefore, Randolph proposed the Spatial Time Division Multiple Access (STDMA) [20], and the nodes of which are fixed geographically and the programming time are divided into multiple time slots, so that data packet transmissions in the same time slot may be deemed as proceeding simultaneously. Each time slot in STDMA is assigned to a group of sets of data packet transmission behavior.

Signal interference types in the scheduling model
When actual data packet transmission occurs, it is inevitably interfered with by other channels transmitting data at the same time. The most suitable model to describe such a mechanism among the numerous literature is the signal interference model [9], [12], [17], which is mostly adopted by the references mentioned in this paper. When routers have transmission abilities between each other, there exist quantifiable transmission energy, and an index for evaluating the level of interference is the ratio of the transmission energy to the total transmission energy of routers transmitting data in the network at the same time; usually, such ratio has to exceed a default signal interference ratio threshold for proper transmission to occur.

Objectives, constraints, and decision variables of the link scheduling model
Common scheduling objectives of wireless mesh networks may be divided into two types. The first type maximizes the open times of transmission links [12]. Under the above STDMA structure, each link may open at any time slot, and the more times it opens in the programming time, the higher the efficiency of the transmission of the data packets at the node. However, many transmission constraints result from the opening of the links, and thus the scheduling model of this type is the process of opening the link as much as possible under transmission constraints to effectively transmit packets.
The second type of objection emphasizes maximizing the data packet transmission amount [9], [17], [18]. In order to observe the status of packet transmission amount, cost information is added to the nodes [17], [18] where the cost is configured so that higher packet transmission amounts correspond to network statuses with lower total cost. Under such a situation, the model has the following formula: where Ht represents the total network cost at programming time t; some models with gateway designs may require these nodes to transmit large amounts to the gateway. Common restraints in scheduling models may be categorized into node packing restraints [11], [19] and signal interference restraints [1]; the former refers to the inability of a node to carry out too many scheduling operations at one time, such as receiving and transmitting packets simultaneously, transmitting data simultaneously to more than two nodes, or receiving data packets simultaneously from more than two nodes; the latter refers to some transmissions with weaker signal intensities being affected or unable to proceed due to the mutual interference of signals when links transmit data packets in the overall network model.

Dynamic programming scheduling
Papadaki has used dynamic programming to solve wireless mesh network scheduling problems [17], [18]. The logic of the programming is to collect packet holding costs from network nodes and transmit data packets as many as possible from the nodes by minimizing the holding cost. The dynamic programming equation mainly constitutes of two parts, wherein the first part is the optimization formula Vt(St) in below: where t is the programming time slot, St is the status of each node of the network at programming time slot t, Ct(St, Xt) represents the cost resulting from decision variable Xt made at programming time slot t, and Vt(St) represents the minimum total cost achievable from the programming time slot t to the end of the programming time under the network status St. The calculation method is to minimize Ct(St, Xt) and the optimization formula of the next stage Vt+1 (St+1) via the decision variable Xt, and the value of the formula when t is 1 is the optimal value of the objective of the link model. The second part is the formula of the transfer of the network node status of each stage; when the optimization formula Vt(St) determines the optimal decision variable of that stage, such decision variables will change the network status of the next stage via this formula. The formula additionally includes randomly arriving data packets as below: st+1 = (st -r xt + at ) + where r is the number of data packets transmitted by the transmission link per unit time, xt is a Boolean variable and indicates that data packet transmission occurred in the link in the programming time if 1, at is the number of data packets arriving at the link in the programming time, and st is the number of the queuing data packets. It is worthy to note that, in this recursive formula, the location of saved data packets and location of randomly arrived data packets are placed on transmission links and thus it is impossible to determine which node is the source of the data packets of the link or which node is the destination (in fact, such data packets are not transmitted to nodes but to other links in the author's mathematical model). Although the decision variables xt indicates the start and end nodes of the link, yet since the source of the data packets received by other links are collectively represented by the random variable at, the relationship between the generation of the random variable and the data packets received by the link becomes obscured. In other words, it is only known that the number of data packets transmitted by the link is rxt, yet the number of packets received by a node from the link is not certain to be rxt.

Scheduling by ADP or other heuristic algorithms
The works in [17], [18] pointed out that under requirements of conventional dynamic programming algorithms, the extent of each stage shall exponentially increase with the increase in the number of nodes and result in difficulties in the actual execution of conventional dynamic programming. One way to handle such issues is to design approximate mathematical models to estimate the original function in a function parameter environment the same as the original problem. In conventional dynamic programming, the problem includes a cost-oriented function f which usually is the optimization function of the conventional dynamic programming and must include all possible extents of the problem and provide recursive functions according to which each stage status will transfer; an approximate formula is a function complying with ( ) f r  ≈ f, and since the formula should correspond to the characteristics of the original formula to yield an approximate result, the designer must be experienced with the problem in practice and able to provide coarse information of the original formula. After providing sufficient information of the original formula, an effective equation to calculate suitable parameters is required as below: where k represents the parameter subscript in the problem space, rk is the basic function under the parameter subscript k, and θk is the weighting of the basic function k. It was mentioned in the previous subsection that Papadaki [17], [18] used dynamic programming to solve the scheduling problem of wireless network models and heuristic algorithms such as ADP optimal, ADP heuristic, and packing heuristic was adopted to seek to compare with each other for the reduction of computing complexity. The results showed that the performances of ADP optimal are second only to dynamic programming algorithm and largely saves computing time; therefore, such algorithm shall be used in this research and performances will be compared with dynamic programming to observe if effective simulation may be performed as in conventional scheduling models.

Link scheduling for wireless network by genetic
algorithm Badia et al. [2] proposed using genetic algorithms to solve the programming problem of wireless mesh networks. When the programming begins, a population is generated randomly or heuristically, and the algorithm operates in repeated processes on generations in practice; a new population P(t+1) is generated from the old population P(t) in every new generation, and the population improves its internal values by genetic operators imitating the evolution mechanism of nature. The evolution mechanism selects superior individuals from the old population to crossover or mutation by a lower probability, and the selection operator handles the work of the selection of superior individuals; the elites in the individuals of the old population usually survive and are preserved during evolution, while other selected individuals are reassembled to generate offspring by the crossover operator and the offspring is preserved to the next generation. After the optimization objective is decided, the crossover and mutation mechanisms are usually designed to seek optimal solutions under confinements, even though it is not guaranteed that the optimal solution may be sought. Despite this, the genetic algorithm is often treated as the method for seeking "sufficiently good" solutions in practice due to the fast execution; in fact, as long as the given evolution generations are large enough, a genetic algorithm usually yields superior solutions. In work in [2], genetic algorithm effectively schedules wireless mesh network models; however, the solved scheduling model design differs from that of Papadaki, and the performance indices are not the same, and thus may not be compared with dynamic programming as to their scheduling schemes. Therefore, the performances of genetic algorithm, dynamic programming, and ADP are all tested in the new wireless mesh network model designed in this research.

Our Method
This section describes how the wireless mesh network model version is redesigned in this research and how the scheduling algorithms are used. The link relationship formed by every node with neighboring nodes is designed according to the mesh topology network structure, and data packets are gradually transmitted from the start to the end as the programming schedule proceeds; each programming unit carries out a dynamic programming algorithm under the prerequisite of meeting all constraints.

Symbol definition
All variables are defined as follows:

Improved wireless mesh network link scheduling model
In the wireless mesh network link model, common objectives are maximizing the data packet transmission amount and maximizing the open times of transmission links; the common objective of such objectives is to effectively transmit maximum data packets in a given time limit. The transmission link scheduling should be considered to maximize the efficiency of data packet transmission when handling such problems and a wireless mesh network link model that conforms to the topology and signal interference characteristics of the wireless mesh network needs to be designed first. In the model of this research, the time flow mechanism is in accordance with time division multiple access (TDMA), and the approach is to divide programming steps according to time into different time slots; after determining the total programming time slot T, t represents a certain time slot.

Programming information on nodes
Conventional wireless network scheduling model places programming information on links, which is different from reality and easily confuses the affiliation of the data packets. To solve such issues, the present research moves the programming information to the nodes. First, the data packet number information at router nodes is defined, and (i, j) represents the coordinates of the node on a two-dimensional map, and Qt,(i,j) is the number of queuing data packets to be transmitted at slot t at node (i,j) and W(i,j) is the number of transmission links on node (i,j). Subsequently, random signal intensity is defined for the links through which the node transmits data to neighboring nodes, denoted as Gt,w,(i,j), and the transmission link also has a changing data packet transmission rate following the signal intensity, denoted as Rt,w,(i,j). Further, r(i,j) denotes the set of nodes that may transmit data packets to node (i,j) and s(i,j) denotes the set of nodes that may receive data packets from node (i, j); each node may be the basic programming u nit.

Objective considering gateway feature
Conventional wireless network scheduling models usually do not include gateway designs, and thus designed objectives in such link models only focus on how to largely transmit data packets regardless of whether these packets were transmitted to the gateway; common forms are maximizing the data packet transmission amount and maximizing the open times of transmission link. The link model in this research combines the features of the objectives of the above models and adds a feature that transmits packets to gateways; by minimizing the holding cost on nodes, the nodes may transmit packets extensively. As the nodes in the model of this research transmit packets unilaterally towards the gateway (the actual construction method is given in detail in the experiments in Section 4), the effect of extensive packets arriving at the gateway may be achieved by maximizing the packet transmission amount. The objective is given as the following:

Constraints in link model
After considering the characteristics of the wireless mesh network under actual situations, the following constraints appear: 1. Flow Constraints This constraint ensures that when the number of queuing data packets of node (i,j) is smaller than the number of packets that may be transmitted by the link per unit time, only the number of presently queuing data packets may be transmitted at maximum, and when the number of queuing data packets of node (i,j) is larger than the number of packets that may be transmitted by the link per unit time, only the number of packets that may be transmitted by the link per unit time may be transmitted at maximum.
This constraint ensures that every data packet at the start node may be transmitted to the end node; however, the following experiments will not adopt this constraint, but the ratio of data packets arriving at the gateway is used as the performance index.
This constraint ensures that the gateway does not have any packet at the beginning; i.e., it is not treated as the start of data transmission. ,( , ), 0, ( , ) , 0,..., 1 This constraint ensures that the gateway does not transmit any data packet, i.e., a data packet does not leave once it enters.

Direct Compatibility Constraints
In a certain programming time slot, no node may simultaneously transmit data packets to more than two nodes.
In a certain programming time slot, no node may simultaneously receive and transmit data packets.

Use ADP for scheduling in the improved model
After introducing the proposed improved model, an algorithm that effectively executes link scheduling is required. From literature, it is known that the performances of ADP is superior to other heuristic algorithms used in WMN link scheduling, and it largely reduces computing amount and achieves performances similar to the optimal algorithm of dynamic programming by simulation methods; therefore, this research adopts this algorithm. Since the wireless mesh network link model is improved, the programming information location and received packet sources in this research are different from the literature.

Dynamic programming operation logic
Before introducing the algorithm logic of ADP, how dynamic programming logic may be used to solve the problems of the redesigned WMN model should be understood first. In the model of the previous subsection, the focus of programming has already been placed on network nodes. In order to facilitate the execution of dynamic programming and enhance link transmission efficiency, more transmission information is added to transmission links on the nodes. First let Zt represent the transmission status of the mesh network at slot t, where Zt,(i,j),w is a binary variable defined as: ,( , ), 1, if the th transmission link has not transmitted before slot at node ( , ); 0, otherwise.
then Zt,(i,j) may be defined as: When not all transmission links of a node have transmitted data before the programming ends, the node receives a penalty because since link transmission does not create cost in this model, all links should transmit as much as possible without violating constraints. Subsequently, let St be the status of the wireless network at slot t, which equals {Qt, Zt}, and includes queuing data packet number and penalty information of the node. The status of each stage of the WMN may be understood via St, which may be referred to in the dynamic programming optimization process and the status transfer of each stage.
Subsequently, the cost is collected from the information in St, and the purpose of maximizing the data packet transmission amount is achieved by minimizing these costs . Let h(i,j) represent the data packet holding cost of node (i, j), and p(i,j) represent the penalty cost on node (i, j) for any unused transmission link. Up to now, every single node contains four kinds of different information (not including link information on the node), wherein the number of queuing data packets Qt is basic information in the link model, and holding cost h, penalty cost pz, and variable Z for judgment of whether to collect penalty are new information added to the original link model for the convenience of executing dynamic programming. Various costs are denoted as Ct (St, Xt), and the collection method is in below: Subsequently, the recursive formula for the status transfer of each stage is defined; the number of queuing data packets and penalty of transmission links of nodes are included in each of the previously defined stage statuses, and the recursive formula for the number of queuing data packets may be given as: From the equation it can be observed that the number of queuing data packets of the node in the next stage equals the number of queuing data packets in this stage plus the number of received data packets from neighboring nodes and the number of randomly arriving data packets minus the number of data packets transmitted from neighboring nodes close to the gateway. The recursive formula for penalty may be given as: 1,( , ) ,( , ) , (1 ) , Next, the optimization formula Vt(St) is designed for each time slot to determine the decision variable of each time slot, and is given as: This formula takes into account both the holding cost and penalty cost, which includes recursive logic with reversed time and represents all generated costs from programming time t to programming end time T and is given in detail in below:

Experimental results and analysis
In this section, different algorithms are used in the scheduling of the new network model, and the same performance index is used for comparison. In previous literature, the comparison results of dynamic programming, ADP algorithm, ADP optimal algorithm, dynamic programming heuristic algorithm, and packing heuristic algorithm show that dynamic programming has the best performance but is not beneficial for large scale computing, while ADP optimal algorithm has second-best performances but largely saves computing time. Therefore, this research uses these two algorithms along with genetic algorithm to perform scheduling on the new model to compare the results for the purposes of observing whether ADP still effectively reduces computing amount and simulates dynamic programming results in the improved link model and comparing the performances of genetic algorithm and ADP. In the following experiments, link scheduling performances of the three algorithms in a small scale WMN model are compared; as for large scale network link scheduling experiments, since possible statuses of hundreds of nodes at each stage cannot be completely listed, only performances of ADP and genetic algorithm will be compared. Figure 1 shows the network topology generated in the present experiment having 8 nodes and 11 links.

Performances of the algorithms in a small scale WMN model
Parameter settings in the experiment are as below:  Total number of time slots: 9  Arriving amount and their probability of random packets at single node: 0: p = 0 Performance index: Ratio of number of data packets arriving gateway to the total number of data packets where the arriving amount and their probability of random packets at a single node refer to the corresponding probability of randomly arriving packet amount at each node, which has an expected value of 1 in a time slot. Therefore, it may be inferred that when the programming time with a total of 9 slots ends, the total data packet expectation value of this network topology with 8 nodes is 8 × 9 × 1=72. The ADP simulation times and smoothing coefficient are suitable values selected after multiple tests. To compare the three different algorithms, the ratio of the number of data packets arriving gateway to the total number of data packets is uniformly selected to be the performance index; Figure 2 Figure 2 in above shows the ratio of the three different algorithms transmitting data packets to the gateway under time; it may be seen that dynamic programming always has the lowest-cost solution under the optimization principle, and although the performance of ADP is slightly inferior to that of the former, the performance is still superior to a genetic algorithm as long as the parameters are designed properly and the simulation time is sufficient. Figure 35 shows the comparison of all costs of dynamic programming and ADP after the programming ends, where the total cost is the sum of the penalty cost and the holding cost; the genetic algorithm is not included since it does not have a cost calculation mechanism like the costs of dynamic programming. The performances of dynamic programming are still slightly superior to ADP. However, since the performances of the latter are all only slightly inferior to those of the former, it is worthy of continuing using in the following large scale network experiments. The expectation value of randomly arriving packets in a time slot of each node in this topology is 0.1. Therefore, when the programming time with a total of 120 slots ends, the total data packet expectation value of this network topology with 256 nodes is 256 × 120 × 0.1 = 3072. The ADP simulation times and smoothing coefficient are set as in the previous experiment, and the ratio of the number of data packets arriving gateway to the total number of data packets is similarly selected to be the performance index to observe the results of every slot. The experiment results in Figure 4 below show that as the time slots increase, the performance of transmitting data packets to gateways of ADP is gradually superior to that of a genetic algorithm.
To observe when all the data packets in the network may be transmitted to the gateways, randomly arriving data packets are not further added after the programming time ends (T = 120), yet the scheduling activity is allowed to continue operating; at this moment the nodes still transmit data packets to the gateways, but it should be noted that the data packets received by any node now is either from itself or received by other nodes before T < 120. According to Figure 5, all experiment samples of the WMN using ADP scheduling have transmitted all data packets to gateways at t = 127, while the model using a genetic algorithm still cannot achieve 100% gateway transmission rate in all experiment samples up to t = 130.

Conclusion
The characteristics of WMN are nothing but the rapid transmission of data packets by using a mesh structure and the constraints of capacity and interference encountered during transmission. However, due to differences in the emphasized purposes of the designers, many scheduling models contain variations and imperfections, such as the lack of randomly arriving data packets, the lack of gateway designs, or the unrealistic packet information saving location. The present research seeks to add as many as possible various characteristics of WMN into the model and make the models from literature reflect closer to reality. In this paper, the WMN model is improved, and programming information is moved to the nodes. In addition to conforming to reality, all algorithms may still be well applied to perform scheduling, and the effect of the simulation of dynamic programming by ADP is still apparent. The new WMN model comprises gateways to which the packets move, and the queuing of data packets at the nodes in each stage and the status of flow to the next stage may be observed via STDMA mechanism and decision variables of each stage. Finally, the experiments indicate that the performance of using ADP in the new model is superior to that of the genetic algorithm.