Biologically-inspired adaptive routing protocol with stochastic route exploration

Rapid increase in amount of tra c and the number of users of information communication networks requires adaptive routing protocols that can properly respond to unexpected change of communication environment such as rapid and large uctuation of tra c. While distributed routing protocols that use only local state of the network have been expected to suitable for adaptive routing on large scale network, the lack of global information of the network often makes it di cult to promptly respond to tra c changes of the network when it occurs at out of the local scope. In this paper, based on the biologically-inspired attractor selection model, we propose a distributed routing protocol with active and stochastic route exploration. Acquiring current state of the network beyond its local scope by utilizing stochastic nature of the protocol, the routing protocol can e ciently respond to rapid change of tra c demand on the network. In order to avoid destabilization of routings due to the exploration, we introduce a short-term memory term to the governing equation of the protocol. We also con rm that the protocol successfully balances rapid exploration with stable routing owing to the memory term by numerical simulations.


INTRODUCTION
Due to rapid increase of the number and diversity of users of the Internet, temporal variation of trac on the Internet becomes rapid and large recently.While periodic variations of trac such as daily uctuation can be easily handled by routing protocols because these uctuations are predictable, unexpected sudden large change such as the ash crowd [1] often causes congestion and failure of reliable trac control.For example, Football Association World Cup held in 1998.The web site of World Cup received 10 times larger requests than usual during the period [2].In order to keep communication quality on the network, therefore, routing protocols that can adapt to these rapid and large trac variations are much expected.So far, several methods to alleviate ash crowd have been proposed [3,4].Most of these approaches work on application level or using ow control and solution based on routing has not been studied so much.
For adaptation to environmental change in large scale networks, distributed routing protocol is more suitable than centralized protocols.Centralized routing algorithms that use global information of a network, such as OSPF [5], now starts to face diculties in feasible operation due to significantly increasing computational costs and communication overheads accompanied with rapid growth of information networks today.In order to suppress communication overhead and realize feasible routings, distributed algorithms in which operation at each node requires only local information of a network attend much attention recently.
In order to realize distributed and self-adaptive routing protocols, various approaches that inspired by adaptive, scalable, and distributed self-organization nature of biological systems have been proposed.Biologically-inspired approaches for distributed routing protocols include AntNet [6], AntHoc-Net [7], where distributed agents set up ecient paths owing to innate randomness of them and indirect interaction mediated by routers.The routing protocol based on attractor selection model [8] realize adaptive path selection according to environmental change by utilizing underlying mechanism of adaptation in which the E.coli adapts to dierent environment [9].Distributed routing protocols select suitable routes relying only on local information of the network, which allows protocols to realize scalable and autonomous routings.However, due to the lack of global information, these protocols often cannot respond quickly to trac changes occurs on a network if these changes occur at outside of its local scope.For example, the routing with the attractor selection measures communication delay on paths only used for current communication without knowing trac load or channel conditions of non-used paths, which makes it highly dicult to promptly react to, for example, unexpected improvement of channel conditions of non-used paths or trac variation on out of its local scope.
Here, in order to realize distributed routing protocols that combine both stable route selection and adaptation to unexpected environmental changes beyond their local scope, we propose a distributed routing protocol based on the attractor selection model.The proposed protocol has active route exploration mechanism that can adapt to rapid trac variation by infrequently and stochastically changing their paths in order to obtain information of non-used paths.In order to suppress excess apping due to the route exploration and to ensure stable behaviors of the model, our extended attractor selection model has a short-term memory which stores internal states of the model before active explorations.If the exploration gives unsatisfactory results, the internal state of the protocol promptly returns to the memorized state.In order to avoid serious quality deterioration due to packet loop or congestion due to the active route change, we also introduce simple loop avoidance methods to our model.
In this paper, we rst explain the routing algorithm based on biologically-inspired attractor selection model in Section II, and proposed our routing protocol in Section III.Then we describe conguration of our numerical simulation in Section IV.Discussions and conclusions are given in Section V and VI.

BIOLOGICALLY-INSPIRED ATTRACTOR SELECTION
In this Section, we provide the original attractor selection model in biological context and introduce the routing protocol based on the model.Then we propose our extended attractor selection model with stochastic exploration and short-term memory.

original attractor selection model
The attractor selection model is nonlinear dynamical systems that describes adaptive synthesis of nutrients in E.coli depending on surrounding environments [8].In order to compensate missing nutrients of the environment, E.coli adaptively change its gene expression of metabolic network and synthesizes required missing nutrients.It is reported that genes corresponding to synthesis of two dierent nutrients suppress each other and the system reaches, after a transient state, a stable xed point, or an attractor, corresponding to synthesis of one of these nutrients.
There are two attractor ⃗ m = (m, 1 m ) and ( 1 m , m) satisfy d ⃗ m dt = 0 excepting the noise term.The System selects one of these two attractors according to the activity α.When the value of the activity is low, the eect of the noise term is dominant, which makes the system uctuate or start to random walk.If the system reaches preferable attractor due to the random walk, the value of the activity increases and uctuation relatively decreases, which allows the system to stay the novel preferable attractor.

routing algorithm based on the attractor selection model
In order to apply the attractor selection model to a packet routing, the model was extended to a high dimensional system [9].
where M is the number of candidate nodes for next hop.
) takes a high value while the others take a low value.When the solution of Eq.( 3) is in the kth attractor, the node forwards packets to the kth next hop node.
Activity α which represents goodness of the current path at time t is dened as, where l(s) is a metric, for example throughput or round trip time, of selected kth path or next hop node at time s.In this paper, we use one-way delay as the metric l.In order to achieve better performance in packet routing, the system tries to converge on an attractor with higher activity or smaller delay.Each node independently calculates ⃗ m and α, and decides their next hop node in a distributed manner.
The activity is calculated from the delay of only actually selected path during past Tw steps.Therefore, if the same attractor continues to be selected over the past time window Tw steps, the value of the activity becomes to, which makes the system to stick to the current attractor without getting information of other paths.Because of convergence to a local solution and lack of global information, it is dicult to achieve sucient adaptive control.

attractor selection model with active exploration
In order to solve the problem of the routing based on existing attractor selection model, we increase variation of the noise of Eq.( 3) to force the system actively exit from attractors and obtain, or search, information of non-used paths.However, stronger noise disturbs convergence of the system to even desirable attractors and makes whole protocol highly unstable.In order to compensate degraded stability, we introduce a short-term memory to the attractor selection model in order to stabilize the model even when it works under a stronger noise.If the system reaches an attractor with lower condition, or activity, by the stochastic active exploration, it return to a memorized state, or an attractor which the system stayed before the exploration while it moves to the new attractor if the new state gives higher activity than before.
The proposed model is given as, where K is the constant characterizing strength of the memory term, τσ is the rate constant, xi represents the memory for mi.The noise strength σ is also a variable in our protocol that increases according to the following equation while the system continues to stay on an attractor.
which allows the system uctuate and change attractor or selected path easily even if the activity remains high.We call this temporal change of attractor "exploration".In order to suppress excess instability or ensure a stable transition after route exploration, we force the value of noise strength σ to δ(≒ 0) just after explorations and recover it up to a constant strength σ * again according to Eq.( 6).
The memory of ⃗ m is denoted as ⃗ x that develops as, Similar to the forgetting curve of human's short-term memory [10], we use the exponential dependency of xi on mi.The fourth term on the right side of the Eq.( 5) represents attracting force from the memory state.The term is small when the system stays at good state, or the activity is high.However, the eect of memory becomes negligibly large when the system reaches a poor attractor, α < 1, due to active explorations.Then the system is pulled back to the memory state by the attraction force.
As with the traditional routing based on attractor selection, activity α id dened as a ratio between metric l of the current and past attractors as, ) where β is the memory of measured metrics, Because time evolution of the noise strength and memories, Eq.( 6), (7), and ( 9) obey similar linear equation with dierent rate constants, we can easily adjust these constants by considering balance between frequency of exploration and speed of convergence to memory states.

ROUTING PROTOCOL BASED ON THE PROPOSED MODEL
In this section, in order to apply the proposed model to the path selection, we explain three things that we have not yet described, implementation of the model to each node, measurement of metric, or communication delay, and loop avoidance.
Each node has dynamics of the attractor selection corresponding to packet destination for each.Therefore, in the network consisting N nodes, there are Metric l, or one-way delay from a source node to a destina-tion node is calculated as the sum of communication delay including transmission delay, queuing delay, and propagation delay.In order to measure the one-way delay along a path, each node sends control packets to destinations per control period Tc.A node that receives the control packet derive the delay by subtracting generating time of the control packet from current time, and returns a feedback packet the source node.When feedback packet reaches the source node, variables of attractor selection model such as state values ⃗ m and activity α are updated.If the source node cannot receive feedback packet because of packet lost, α decreases by half without updating metric l.
If packets start to loop among several nodes, it may result in serious degradation of performance of routing protocols.To avoid loop as simply as possible, we introduce two loop avoidance mechanisms into the proposed protocol.First, in order to avoid loop between adjacent nodes, we make each node remove adjacent nodes who sends packets to themselves from candidate next hop nodes of route selection, which is easily realized by using hello packets that have pairs of destination node and next hop node in routeing table (line 23 to 27 of the Algorithm 1).Second, in order to reduce inuence of loop consisting more than three nodes, we make nodes send a packet to a random next hop node if the packets has passed before based on its routing table.

EVALUATION WITH A NETWORK SIM-ULATION
In this section, we rst explain our network simulation environments and then provide our main results with comparison to several existing methods.

simulation environment
We evaluate the proposed protocol in routing on a queuing network.The network is a connected random graph consisting N = 36 nodes which is generated by the Waxman model [11].Each pair of adjacent node is connected by 1Mbps bidirectional link and packet takes 1ms xed propagation delay to pass the link.Each link has buer with size of 100Kbyte.Packets are discarded when their hop count reaches to TTL=10 hops or queue length exceeds the buer size.Each node generates new packet at a given packet generation rate.The packet size is distributed from 64 to 1500 Bytes based on the exponential distribution.In this simulation, we regard metric l as average of communication delay of all data packets and do not model the control packet explicitly.Control period Tc = 100s.

results of the network simulation
We show adaptation of the proposed model to rapid trac variation.Each node generates packets to all other nodes in 10Kbps packet generation rate.In the middle of the simula- We study two scenarios of the trac variation; one is the download scenario in which a node increases trac to all other nodes and another is the upload scenario in which opposite trac, or trac to a node from all other nodes, increases.Figure .1 shows the average delay over all simulation of the proposed model comparing to MINHOP that always selects the shortest path, and GREEDY in which each node forwards packets to a path with minimum delay at each period.PROPOSAL(k = 0) explores non-used path owning to strengthen noise but doesn't have memory or attracting force to a memorized state.
Figure.1(a) shows average delays in the download scenario.Until time 5000 three methods other than PROPOSAL(k = 0) achieve similar better performance.When trac variation occurs, the proposed protocol starts to change their routing table by stochastic and active exploration and each node tries to nd better attractor in autonomous distributed manner, which nally make the system converge to the solution that gives smaller delay.MINHOP that continues to select the same shortest path converges to state thatnally increase average delay.In GREEDY method that each node changes route selsh independently, delay is also relatively large because it leads the system to continuing apping.Figure.1(b) shows the upload scenario.We can see that the basic tendency of the results is similar with download scenario while the proposed model takes longer time to reach convergence in the upload scenario and overall average delay is a litter larger than the download scenario.This difference is explained as packets from many other nodes that are concentrating on around the destination node in the upload scenario.Due to the concentration, transient rapid increase of average delay occurs in the upload scenario.We will discuss this in Section V.
Figure .2shows results of similar evaluation on a large network consisting N = 128 nodes, where available bandwidth of all links are adjusted to 10Mbps and each node generates packets in 100Kbps.Each method perform similarly with the results of N = 36, which may suggests that the proposed protocol is scalable for network size.Systematic evaluation of performance of the proposed protocol for ranges of size is an important future subject.
Figure .3shows the average delay for dierent values of K in the download scenario.The average delay takes smaller value over a wide range excepting K = 0.This implies that ne tuning of the model parameter K is not necessary.When K = 0, due to stochastic and active exploration,the system cannot converge to even desirable attractor and starts to random walk.The result shows that nite values of K is eciently suppress this possible instability of the proposed protocol.or small rate constant τσ, noise strength becomes large and intervals between active route change become narrow.This frequent stochastic exploration allows the system to adapt to trac variation more rapidly and decrease average delay as shown in the donwload scenario.Figure .4(a).However, excessive stochastic exploration decreases total performance of the protocol even if the memory term tries to stabilize the system especially for stationary scenario.In the stationary scenario Fig. 4(b), though the shortest path is optimal, the proposed protocol sometimes tries to use detour route for exploration and decreases performance a little.Therefore, it is important to choose the noise parameters considering tradeo between performance at stationary trac and adaptation to trac variation.

DISCUSSION
The proposed protocol can robustly responds to trac variation and achieves higher performance, or short average delay, when large and rapid trac changes occurs.However the protocol still sometimes leads transient increase of communication delay as shown in small spikes in Fig. 1(b).In order to further improve performance of the protocol, it may be important to suppress or decrease such a transient increase of communication delay.The cause of this increase of delay is temporal congestion along a path or buer overow on a router induced by stochastic route change for route exploration.If a node changes its next hop node stochastically to use a link whose utilization is already high or a node that already relays many packets, inow to the next node exceeds bandwidth and leads rapid degeneration of performance i.e. sudden increase of communication delay along the node.While the result of exploration is immediately discarded and the next hop node is returned to the memorized state in the next control period immediately, communication that shares this link can be aected.It must be an important future subject to prevent route changes that can lead such sudden congestion by utilizing additional metric such as, for example, link utilization.

CONCLUSION
In this paper, we proposed a distributed routing protocol based on the attractor selection that can adapt to rapid and large trac variation without constantly referring global information of networks.Our protocol tries to search better solutions, or routes, over its local scope by using enhanced noise term.Simultaneously, potential instability of the routing protocol due to the active exploration is safely suppressed because the proposed system holds their state variables into their memory variables to rapidly return to the original state if results of explorations are worse.We showed validity of the proposed protocol in various scenarios of trac change and study suitable parameter settings of the model.

Figure 1 :
Figure 1: time courses of average delay

Figure 2 :
Figure 2: time courses of average delay

Figure. 4 Figure 3 :Figure 4 :
Figure.4 shows average delay for values of noise parameters σ * and τσ in the scenario with and without trac varia- N (N − 1) attractor selection models are running on the whole network principally.Each node forwards packets based on a routing table that has pairs of destination and next hop decided by the attractor selection for the destination.Algorithm 1 describes how to update routing table on a node P to a node Q.Each node sends Hello packet to adjacent nodes per hello interval time to conrm existing connections.If the destination of a packet is an adjacent node, the packet is directly forwarded to the router.Since route selection is not necessary for adjacent nodes, each node which has r adjacent nodes runs N − r dynamics of attractor selection in fact.Algorithm 1 update routing table on node.P to node.Q 1: if P has a direct link to Q then 2: next-hop-node ← Q