Analysis of Rate-Based Pull and Push Strategies with Limited Migration Rates in Large Distributed Networks

In this paper we analyze the performance of pull and push strategies in large homogeneous distributed systems where the number of job transfers per time unit is limited. Job transfer strategies which rely on lightly-loaded servers to attract jobs from heavily-loaded servers are known as pull strategies, whereas for push strategies the heavily loaded servers initiate the job transfers to lightly loaded servers. To this end, servers transmit probe messages to discover other servers that are able to take part in a job transfer. Previous work on rate-based pull and push strategies focused on the impact of the probe rate on the mean job response time. In this paper we also limit the overall migration rate and show that any predeﬁned migration rate can be matched by both the rate-based pull and push strategies. We present closed form formulas for the mean response time (as a function of the allowed probe and migration rate) and validate their accuracy by simulation. We also introduce and analyze a new pull strategy and show that under high loads it is superior to the push strategies considered, while the push strategies oﬀer only a very limited gain for medium to low load scenarios.


INTRODUCTION
In order to optimally use the available resources in a distributed network it is desirable to be able to dynamically relocate jobs among a large number of processing nodes.Jobs may enter the network via one or multiple central dispatchers (e.g., [4,10,12,14,15]) or via the processing nodes themselves (e.g., [2,3,8,13]).A central dispatcher will distribute the jobs among the nodes using some load balancing algorithm.In a more distributed approach, the nodes themselves will arrange for jobs to relocate after they are scheduled.Two approaches are common: push and pull.In a push variant (or work sharing) highly loaded nodes attempt to find lightly loaded nodes to migrate jobs to.Pull variants (or work stealing) reverse the roles, so that lightly loaded nodes try to attract work from the highly loaded nodes.
Several authors studied the performance of push and pull strategies.A comparison for a homogeneous distributed system with Poisson arrivals and exponential job lengths was presented in [1,2] and extended to heterogeneous systems in [9,11].These studies showed that the pull strategy is superior under high load conditions, while the push strategy achieves a lower mean delay under low to moderate loads.
Nodes typically communicate by means of probe messages, exchanging information such as queue length.For simplicity we assume that sending/receiving probe messages is instantaneous and does not incur an extra computational or bandwidth cost.When a node wants to push or pull a job, it probes a random other node to see if a transfer between the nodes would be allowed.
Under a traditional pull or push strategy a server sends a maximum of Lp probes the instant its last job completes or the instant a job arrives when the server is already busy [1,2].The fraction of queues sending probe messages is different, and as a result pull and push strategies achieve a different overall probe rate for the same load of the system.This makes a performance comparison biased, as sometimes the strategy with the higher probe rate is best [5].
In [5] rate-based pull and push variants are introduced that can match any predetermined probe rate R, allowing the comparison of pull and push strategies when they use the same number of probes.In these variants, probes are no longer sent at job arrival or completion times but at a fixed rate r as long as the server is idle (for pull) or has jobs waiting (for push).The main result in [5] showed that the rate-based push strategy results in a lower mean delay if and only if under the so-called infinite system model and that a hybrid pull/push strategy is always inferior to the pure pull or push strategy.
The paper is structured as follows.Section 2 summarizes the rate-based strategies considered in this paper.In Section 3 we briefly summarize earlier work concerning ratebased pull and push strategies, and introduce an overall migration limit M for these strategies.Also, we derive an expression for the corresponding maximum probe rate r both|M for the rate-based push and pull, and rewrite the mean delay in an equivalent form.In Section 4 we adapt the max-push strategy to match M by finding the corresponding probe rate r mp|M , after summarizing earlier work.Section 5 considers pull strategies with T > 1, and introduces the new conditional-pull strategy.It is shown that the conditionalpull is equivalent to the max-push strategy in case there is no probe limit R and only a migration limit M .In addition, the infinite system model describing the evolution of the conditional pull strategy is numerically validated, and argued to be the proper limiting process as the system size tends to infinity.Finally, we compare the mean delay of max-push and conditional-pull in Section 6.

RATE BASED STRATEGIES
We consider a continuous-time system of N queues, where each queue has a single server and infinite buffer.Each queue operates under Poisson job arrivals with rate λ < 1, and exponential service time with mean 1. Jobs are processed in a first-come-first-served order.
Traditional strategies send a maximum of Lp probes the instant a server's last job completes or the instant a job arrives when the server is already busy.In contrast, under rate-based strategies probes are no longer sent at job arrival or completion times but at a fixed rate r as long as the server is idle (pull) or has at least T jobs waiting (push).More formally, probe messages are transmitted by a server according to an interrupted Poisson process with rate r.
The strategies considered in this paper can be summarized as follows: 1. Rate-based Push: As soon as the queue length exceeds T , a server starts to generate probe messages according to a Poisson process with rate r.Whenever the queue length drops below T , this process is interrupted until the queue length exceeds T again.The node that is probed is selected at random and is only allowed to accept a job if it is idle.
2. Rate-based Pull: Whenever a server is idle it generates probe messages according to a Poisson process with rate r.This process is interrupted whenever the server is busy.The node that is probed is selected at random and is only allowed to transfer one of its jobs if its queue length exceeds T .

Max-Push:
The instant a new job arrives at a queue with length T , probes are sent at an infinite rate.When λ < 1 this corresponds to stating that the job is instantaneously transferred to an empty server.A server with T jobs in its queue, generates probe messages according to a Poisson process with rate r.Whenever the queue length drops to T − 1, this process is interrupted as long as the queue length remains below T .The node that is probed is selected at random and is only allowed to accept a job if it is idle.
4. Conditional-Pull: Whenever a node is idle, the node will generate probe messages according to a Poisson process with rate r.This process is interrupted whenever the server becomes busy.The probed node is selected at random and the probe is always successful if there are at least T jobs waiting to be served, and successful with some probability p (matching M , see (26)) if there are exactly T − 1 jobs waiting to be served.
We do not consider hybrid strategies, which combine both push and pull behavior.These were proven to be inferior to a pure push or pull strategy when T = 1 [5, Theorem 4].

PULL AND PUSH STRATEGIES
Infinite system models and closed form solutions for both pull and push strategies were introduced in [5] and [6].Before introducing new constraints and strategies, we briefly summarize the main findings of [5] and [6].
The evolution of both the rate-based pull and push strategy under the infinite system model is described by a set of ODEs denoted as d dt x(t) = F (x(t)), where x(t) = (x1(t), x2(t), . ..) and xi(t) represents the fraction of the number of nodes with at least i jobs at time t.
From [6, Theorem 2 and 3], it is known that d dt x(t) = F (x(t)) has a unique fixed point π = (π1, π2, . ..) with i≥1 πi < ∞ that is a global attractor, given by πi This fixed point is used in conjunction with Little's Law in [6, Corollary 1] to formulate the mean delay D both of a job under the push or pull strategy: From the relationships R = (1 − π1)rpull|R and R = r push|R πT +1, we find and It follows that whenever R > λ T +1 /(1−λ T ), the rate r push|R can be chosen arbitrarily large (i.e., r push|R = ∞).

Limiting the Overall Migration Rate
When the overall migration rate is limited, the choice of r must satisfy this constraint.We first indicate how to set r to match M , and rewrite the formula for the mean delay.
Then we show whether R or M is the strictest constraint for a given load λ.
Theorem 1.Both rate-based pull and push strategies match a predefined migration rate M by letting the probe rate r = r both|M , with r both|M : For this setting, both strategies achieve the same mean delay.
Proof.The relationship (6) readily follows from the formulation of the overall migration rate for both rate-based strategies: From a push perspective this equation describes the fraction of nodes with queue length larger than or equal to T + 1 1)) sending probes at rate r, succeeding with probability (1 − λ).For a pull strategy the overall migration rate is expressed as the fraction of empty queues (1 − λ) sending probes at rate r and succeeding when probing a queue of length T + 1 or longer (πT Theorem 2. The mean delay of rate-based pull and push strategies can be expressed as The probe rates r imposed by either the probe limit R = 1 for push (dot-dashed) and pull (dashed), or migration limit M = 1/8 (full), for T = 1, 2, 3. Note that r pull|R is independent of T .
The previous theorem shows that the improvement in mean delay compared to a standard M/M/1 queue, can be expressed as a migration frequency (M/λ) times a migration gain (T + λ/((1 − λ)(1 + r))).The migration frequency denotes how many migrations per job take place on average.The migration gain quantifies the number of places in the queue the migrating job skips.All migrating jobs skip at least T places by construction of the strategy, and skip more places depending on the queue length of the job sender.The average number of places skipped above T equals the average number of customers in an M/M/1 queue with service rate 1 + r(1 − λ), which equals λ/((1 − λ)(1 + r)).
Both the overall migration limit M and the overall probe limit R impose a maximum on r.In case R ≤ M , all probes are allowed to generate a migration, so r will only be constrained by R. In any practical setting there will be more probes allowed than migrations, and the probe rate will be constrained by R or M depending on λ.An overview of the probe rates matching R or M , for both push and pull strategies with T = 1, 2, 3, is given in Figure 1.
To determine the range of λ in which each constraint is most strict, we determine the intersection of r both|M with r push|R and r pull|R .
Lemma 1.The probe rates r push|R and r both|M intersect at λ = 0 and 1 − M/R only, and both rates are positive for The proof is given in Appendix A of [7].
Theorem 3.For the rate-based push strategy r should be set as follows in order to respect both the probe rate R and migration rate M : r can be arbitrarily large.
r can be at most r both|M .
r can be at most r push|R .
• If τ > υ, r can be at most r both|M if λ < 1 − M/R and at most r push|R otherwise.
• If τ ≤ υ, r can be at most r push|R .
The proof is given in Appendix B of [7].
The proof is given in Appendix C of [7].For T = 1, the unique root λT of Lemma 2 reduces to λ1 = M + M/R.However, there seems to be no closed form expression for general T .
Theorem 4. For the rate-based pull strategy r should be set as follows in order to respect both the probe rate R and migration rate M : ), then r can be at most r pull|R .
• If the migration limit is sufficiently low (M < R 1+T R ), r can be at most r pull|R if λ < λT , and at most r both|M otherwise.
The proof is given in Appendix D of [7].

MAX-PUSH
The rate-based push is unable to reach an overall request rate higher than λ T +1 /(1 − λ T ) for any T .When the overall probe limit R exceeds this value, it is possible to use the remaining request rate by using the max-push variant as introduced in [6].This strategy lets nodes with a queue length of T send probes at a finite rate r mp|R , and migrates all new arrivals to queues with length T by sending probes at an infinite rate until an empty server is found.As λ T +1 /(1− λ T ) is an increasing function in λ and decreasing in T , the unique solution for λ to λ T +1 /(1 − λ T ) = R is increasing in T .Therefore, there is a unique T > 1 satisfying For this T , the evolution of the max-push strategy can be described by a set of ODEs d dt x(t) = G(x(t)), where x(t) = (x1(t), x2(t), . ..) and xi(t) represents the fraction of the number of nodes with at least i jobs at time t.
Theorems 7, 8 from [6] show that the set of ODEs has a unique fixed point π = ( π1, . . ., πT ) that is a global attractor, and can be expressed as for 1 ≤ i ≤ T .The mean delay Dmp of a job under the max-push strategy is given by [6, Corollary 3]: For the max-push strategy the overall probe rate R equals as the instantaneous transfer of an arrival to a queue with T jobs requires 1/(1 − λ) probe messages on average.Therefore, a predefined overall probe rate R can be matched by setting where 0 ≤ r mp|R < ∞ for λ T +1 /(1 − λ T ) ≤ R < λ T /(1 − λ T −1 ).

Limiting the Overall Migration Rate
As the rate-based push strategy cannot exceed the probe rate λ T +1 /(1 − λ T ), it is unable to exceed an overall migration rate of (1 − λ)λ T +1 /(1 − λ T ).It follows that when M > (1 − λ)λ T +1 /(1 − λ T ), queues with a length of at least T +1 can probe at an arbitrarily high rate without exceeding the migration limit M , effectively reducing all queues to a length of at most T .Queues with length T can then send probes with a finite r to match M .In other words, in order to match M (instead of R as in the previous section), set T such that and determine the probe rate r mp|M when the queue length equals T by the following theorem: Theorem 5.The max-push strategy matches a predefined migration rate M by letting the probe rate r = r mp|M with When matching M this way, πi for i ≤ T reduces to Proof.The relationship (15) follows from the formulation of the overall migration rate.The migrations of new arrivals in queues with length T are given by πT λ.The migrations resulting from a successful probe sent by queues with length T are given by πT r(1−λ).Probes are successful if they locate an empty server, which they do with probability 1 − λ.Therefore, the overall migration rate can be expressed as The reduction of πi to (16) follows from substitution of ( 15) in (10), and shows the improvement over an M/M/1 queue directly.Theorem 6.For the max-push strategy r and T should be set as follows in order to respect both the probe rate R and migration rate M : • If λ < 1 − M/R, T must be chosen according to (14) and r can be at most r mp|M .
• If λ > 1 − M/R, T must be chosen according to (9) and r can be at most r mp|R .
The proof is given in Appendix E of [7].
Theorem 7. The mean delay of the max-push strategy can be expressed as with α = πT λT and β = πT r(1 − λ)(T − 1).When r = r mp|M , the mean delay Dmp reduces to Proof.The improvement in mean delay compared to a standard M/M/1 queue, can be expressed as a migration frequency (Mmp/λ) times a migration gain.The migration frequency denotes how many migrations per job take place on average.The migration gain quantifies the number of places in the queue the migrating job skips.The fraction of migrating jobs arriving at a queue with length T ( πT λ/Mmp) skip T places in the queue.The fraction of migrating jobs from queues with length T , being πT r(1 − λ)/Mmp, skip T − 1 places in the queue.Hence, the migration gain is (α + β)/Mmp.
The reduction to D mp|M is found by applying Little's Law to the expression for π in (16), and shows the improvement over an M/M/1 queue explicitly.The probe rate r is constrained by both R and M (full lines).The delay shown in dashed lines is achieved when there is no migration limit, and only the probe limit is in effect.Choosing T = 1 is no longer optimal when a maximum migration rate is imposed.

CONDITIONAL PULL
When only considering a maximum allowed probe rate R, the optimal choice for a pull strategy is to let T = 1 [6, Theorem 5].This is no longer the case when taking a maximum allowed migration rate M into account, as shown in Figure 2. Intuitively, when the migration limit is small, it is best to pull jobs from longer queues only, resulting in a lower mean delay.
To reduce the mean delay of the rate-based pull strategy, we introduce the conditional pull strategy that can match both R and M .Empty servers send probes according to an interrupted Poisson process with rate r.Under the conditional pull strategy, empty nodes always accept jobs from queues with length of at least T + 1 and also accept jobs from a queue with length T with some probability p.This strategy relies on the choice of p to match the migration rate M , and lets the probe rate r be determined by R, i.e. r = r pull|R = R/(1 − λ).Thus, both R and M are matched, this is in contrast with the previous strategies, where r was always chosen as large as possible without exceeding R and M .
First we note that one can easily see that for λT , being the unique root defined in Lemma 2, λT < λT +1 (as M/((λ with λ0 = 0. To analyze the response time of a job under the conditional pull strategy we introduce a set of ODEs for 1 < i < T , and for i ≥ T , where for i = T . The next theorem shows that this set of ODEs has a unique fixed point with i≥1 πi < ∞.In Appendix F of [7] we briefly argue why this fixed point can be used to approximate the queue length distribution of a node as the number of nodes becomes large.The argument is similar to the one used in [5].We also validate the accuracy of this approximation by simulation in Section 5.1.Theorem 8.The set of ODEs d dt x(t) = H(x(t)) has a unique fixed point π = (π1, π2, . ..) with i≥1 πi < ∞.The fixed point can be expressed as: for 1 ≤ i ≤ T , and for i > T as Proof.Assume π is a fixed point with i≥1 πi < ∞, meaning Hi(π) = 0 for i ≥ 1, where H(x) = (H1(x), H2(x), . ..).When i≥1 πi < ∞, we can simplify i≥1 Hi(π) = 0 to λ − π1 = 0. Hence, π1 must equal λ.The expressions for πi then readily follow from the conditions Hi(π) = 0, for i ≥ 1.
Theorem 9. A predefined overall migration rate M can be matched by setting p = p cp|M , with .
When matching M by setting p = p cp|M , πi reduces to for i ≤ T .
Proof.The fraction of empty queues (1−λ) send probes at rate r.Probes are successful with probability 1 if they locate a queue with length at least T + 1 (πT +1).Probes are successful with probability p if they locate a queue with length equal to T (πT − πT +1).In other words: from which (26) follows by algebraic manipulation.The reduction of πi to (27) is found by substituting (26) in (24).
Theorem 10.The mean delay Dcp of a job under the conditional pull strategy equals with Proof.The improvement in mean delay compared to a standard M/M/1 queue, can be expressed as a migration frequency (Mcp/λ) times a migration gain (α − β).The fraction of migrating jobs where the job is pulled from a queue with length at least T + 1, (r(1 − λ)πT +1/Mcp) skip α places in the queue: The same remarks as in Theorem 2 apply.The other jobs ((r(1 − λ)p(πT − πT +1))/Mcp) are pulled from a queue with length equal to T , thus skipping exactly T − 1 places.In other words, the migration gain can be expressed as: Mcp , which can be rewritten as α − β.
Figure 3 shows the mean delay of the conditional pull strategy.The dots represent λT , i.e., the intersection points of r pull|R and r both|M .The conditional pull strategy achieves a lower mean delay compared to the rate-based pull strategies with T > 1, as it transfers more jobs.
Theorem 11.When R = +∞ and M is finite, the maxpush and conditional pull strategies have the same stationary queue length distribution.
Proof.When there is no probe limit R, the parameter r is allowed to be arbitrarily large for the conditional pull strategy.In this case the maximum queue length will be T as limr→∞ πi = 0 for i > T , see (25).We therefore know from Theorems 5 and 9 that both the max-push and conditional pull strategy have the same queue length distribution when only matching M , if they use the same T .What remains to be shown is that both strategies make use of the same T .
Recall that λT was defined as the solution in (0, 1) of r pull|R − r both|M , that is, The left hand side tends to infinity as R tends to infinity.Hence, r both|M must tend to infinity, meaning λT is the solution to M = (1 − λ)λ T +1 /(1 − λ T ).The λT 's are thus exactly the M values where the max-push strategy changes its T value, see (14).Hence, both the conditional pull and max-push strategy choose the same T .

Model Validation
We validate the infinite system model for the conditional pull strategy by comparing the closed form results of Theorem 10 with time consuming simulation results for systems with a finite number of nodes N .The infinite and finite system model only differ in the system size.Hence, the rate r and probability p in the simulation experiments is independent of N and was determined by using the expression for p from Equation (26) and r = R/(1 − λ).Each simulated point in the figures represents the average value of 25 simulation runs.Each run has a length of 10 6 time units (where the service time is exponentially distributed with a mean of 1 time unit) and a warm-up period of length 10 6 /3 time units.
Table 1 compares the mean delay in a finite system with N nodes with the mean delay in the infinite system model under the conditional pull strategy with R = 1 and M = 0.1 for N = 25, 50, . . ., 1600 and λ = 0.5, 0.65, 0.7, 0.75 and 0.8.For each combination of N and λ we also show the relative error.The error clearly decreases as N grows, and is worse for larger λ values.
The observed overall migration rate in the simulation is strictly lower than the predefined M , meaning less jobs will be transferred than anticipated.Hence, the mean delay in the simulation experiments is pessimistic.This error is in part due to the choice of p, which was determined using (26).This choice relies on the infinite system model whereas we are now studying a finite system.The relative error in the observed overall migration rate is nearly load-insensitive and decreases linearly as the system doubles in size, as shown in Table 2.

PUSH VERSUS PULL STRATEGIES
We compare the performance of the max-push and the conditional pull strategies with a predefined overall probe limit R and migration limit M (using Theorems 7 and 10).The parameter T is determined by the load λ, as each strategy is only defined for a specific T given any λ (see ( 9), ( 14) and (20)).For the max-push, the value for T and r is chosen to match the strictest constraint of either R or M depending on the load (see Theorem 6).For the conditional pull all idle servers probe with rate r = R/(1 − λ), and p is chosen to match M (see Theorem 9).
The mean delay of the max-push and conditional pull strategy with M = 0.1 and R = 0.4 is shown in Figure 4.The max-push strategy is limited by the probe limit when λ > 1 − M/R and by the migration limit when √ M < λ < 1 − M/R.The mean delay of the push strategy is one in case λ < √ M , as all newly arriving jobs at a busy server can be migrated instantaneously to an empty server without violating the R and M constraints.For the conditional pull strategy the limiting factor is R when λ < M + M/R, and M for λ > M + M/R.
The intervals where both strategies are constrained by M do not always overlap, i.e.
M + M/R can be larger than 1 − M/R, as is the case for R = 1 and M = 0.3.When M + M/R < 1 − M/R both strategies transfer the same number of jobs when M + M/R < λ < 1 − M/R.However, the max-push will outperform the conditional pull as the average migration gain is larger.This is not unexpected as the max-push strategy avoids that queues become larger than T , whereas queues with a length exceeding T exist for the conditional pull as it only sends random probes at a finite rate.As expected from Theorem 11, the difference in performance between max-push and conditional-pull becomes smaller when increasing R, as shown in Figure 5.As the empty queues send probes with rate r = R/(1 − λ), they are allowed to send more probes as R increases.This increases the odds that a long queue is probed, thus lowering the mean delay.This can also be observed by looking at the values for T .By increasing R, a larger value for T can be used for the same load.This requires that jobs are pulled from longer queues, increasing the migration gain per transfer.
In conclusion, whenever the maximum allowed probe rate R clearly exceeds the maximum allowed migration rate M (which is the case that is mainly of practical interest), the pull strategy is either clearly superior (for large λ) or has a similar performance to the max-push strategy (for medium to low λ).

)Figure 1 :
Figure1:The probe rates r imposed by either the probe limit R = 1 for push (dot-dashed) and pull (dashed), or migration limit M = 1/8 (full), for T = 1, 2, 3. Note that r pull|R is independent of T .

Figure 2 :
Figure2: The mean delay of the pull strategy for T = 1, ..., 4. The probe rate r is constrained by both R and M (full lines).The delay shown in dashed lines is achieved when there is no migration limit, and only the probe limit is in effect.Choosing T = 1 is no longer optimal when a maximum migration rate is imposed.

Figure 3 :
Figure 3: Showing the mean delay of the pull strategy with T > 1, respecting a migration limit Ṁ .The conditional pull variant is shown in dashed lines.

Figure 4 :Figure 5 :
Figure 4: Mean delay of the max-push and conditional pull strategies, with R = 0.4 and M = 0.1.