Optimal sleep-state control of energy-aware M / G / 1 queues

We study the problem of optimally controlling the use of sleep states in an energy-aware M/G/1 queue. In our model, we consider a family of policies where the server upon becoming idle can wait for a random period before entering, potentially randomly, any of a finite number of possible sleep states to save energy. The server becomes busy again after a possibly random number of jobs have arrived. However, jobs are served only after a random setup time. This kind of an energy-aware queuing system has been analyzed in recent papers under specific assumptions regarding the cost metrics and the distributions of the random variables. In this paper, we consider an essentially more general model. Notably we show that the optimal control of the idle time and sleep states is deterministic and does not benefit from randomization: either the system only uses the idle state and no sleep states, or the idle state is not used at all and the server immediately goes to some fixed sleep state and waits until a fixed number of jobs have arrived before starting the setup. We prove this result for two popular cost metrics, namely weighted sum of energy and response time (ERWS) and their product ERP.


INTRODUCTION
An increasing demand for green ICT has inspired the queueing community to consider energy-aware queueing systems.In many cases, it is no longer enough to optimize just the performance costs but one should also take into account the energy costs.An idle server (waiting for an arriving job to be processed) in the server farm of a typical data center may consume as much as 60% of the peak power.From the energy point of view, such an idle server should be switched off until a new job arrives.However, from the performance point of view, this is suboptimal since it typically takes a rather long time to wake the server up.Thus, there is a clear trade-off between the performance and energy aspects.The performance aspects of similar vacation models have been analyzed in, e.g., [18,5,15].
The two main metrics used in the literature to analyze the performance-energy trade-off in energy-aware queueing systems are ERWS [1,4,3,19,2,16,9] and ERP [8,17,12,11,13,6].Both of them are based on the expected response time, E[T ], and the expected power consumption per time unit, E[P ].The former one, ERWS, is defined as their weighted sum, and the latter one, ERP, as their product, In this paper, we focus on the optimal control of a single energy-aware server.We assume that jobs arrive according to a Poisson process and service times are IID with a general distribution.In addition to the off state, we assume that the server has multiple intermediate sleep states, for which the deeper the sleep state, the smaller the power consumption but also the longer the setup delay.The possible control actions are as follows: (i) When the server becomes idle, it will start a timer denoted by I, which is an IID random variable with a general distribution (including even the deterministic special cases I = 0 and I = ∞), and wait until either a new job arrives or the timer expires.If there is an arrival before the timer expiration, the server will immediately start a new busy period and no further control actions are possible until the server next becomes idle again. 1ii) However, if the timer expires before a new arrival, the server will independently and randomly choose one of the possible sleeping states, say i, and a switch-on threshold, say k, from a given distribution p = (pik).The server will stay in the chosen sleep state i until the queue length reaches the chosen threshold k.At that time, the server is switched on so that after a setup delay denoted by Di, which is an independent random variable with a general distribution depending on i, the system will start a new busy period and no further control actions are possible until the server next becomes idle again.
A control policy in this system is defined by giving the distribution of the idling time I and the sleep-state/threshold distribution p.The target is to find the optimal distributions that minimize the chosen cost metric.
Our purpose is to unify and generalize some results presented in two recent papers, [6] and [14].Gandhi et al. [6] considered this model, however, without an idling option (so that only I = 0 and I = ∞ are possible alternatives) and assuming exponential service times and deterministic setup delays.They showed that the optimal policy for the ERP metric is deterministic with threshold k * = 1 (i.e., there is i * such that pi * ,1 = 1).In this paper, we prove that the optimal policy still remains deterministic even under the idling option and for any distributions for service times and setup delays.We also show that the optimal threshold may not necessarily be k = 1 for the ERP metric when general service and setup times are considered.In addition, we prove that the optimal policy is deterministic also for the ERWS metric with the optimal threshold being not necessarily k = 1.
Maccio and Down [14] also considered this model but they restricted themselves to a single sleep state and deterministic rules to choose the corresponding threshold k.They showed, however, explicitly only for the ERWS metric and assuming exponential service times and setup delays, that the optimal idling time is either I = 0 or I = ∞.In addition, they gave a heuristic argument for the claim that the result would be valid for a more general cost metric, namely which covers both common metrics, ERWS and ERP.In this paper, we prove that the optimal idling time is, indeed, either I = 0 or I = ∞ for the ERWS and ERP cost metrics even if we allow general distributions for service times and setup delays, multiple sleep states, and randomized rules to choose the sleep state i and the corresponding threshold k.
In addition, we prove that the result is valid for the general cost metric (3) if there is only a single term in the sum, i.e., for the cost metric which is a slightly generalized version of the ERP metric.However, if there are multiple terms in (3), we demonstrate, by constructing a counter-example, that the optimal idling time may be different from I = 0 and I = ∞.
The rest of the paper is organized as follows.In Section 2, we introduce the system model and its analysis is in Section 3. Cost metric optimization is carried out in Section 4, and an extension of idling time considerations is given in Section 5. We give numerical examples in Section 6 and conclude the paper in Section 7.

MODEL
We consider an energy-aware M/G/ All setup delays are assumed to be independent of each other.The (deterministic) power consumption during any setup delay is denoted by Psetup.
As already described in Section 1, the power consumption is controlled by a policy, which is defined by specifying the following two distributions: (i) the distribution for the idling time I (with the special cases I = 0 and I = ∞ included) and (ii) the distribution p = (pik; i ∈ {0, . . ., n − 1}, k ∈ {1, 2, . ..}).Let Πmixed denote the family of all such policies.2 In the sequel, we will also use the following short-hand notations: Recall that when the idling time timer expires in state idle, the following sleep state will be i and the corresponding switch-on threshold k with probability pik.Initially we assume, as in [14], that the idling time timer is reset (to 0) only when it expires.Thus, if the server is idle and a new job arrives before the timer expires, the server moves from state idle to busy and the idling time timer is not reset but the accumulated idling time is remembered for the next time the server becomes idle again.In Section 5, we argue that the optimality results remain the same even if the idling time timer is reset after any idle period.

ANALYSIS
In this section, we derive the mean response time E[T ] and the mean power consumption E[P ] for the policies belonging to Πmixed.The results are given below in Theorems 1 and 2, which generalize the corresponding earlier results given in [6] and [14].
The special case I = ∞ corresponds to an ordinary M/G/1 queue for which Consider now any policy for which E[I] < ∞.In this case the idling time timer expires sooner or later, and every time it expires, the system regenerates itself.
Let C denote a regeneration cycle between two consecutive timer expirations.Let Tbusy, Tidle, Tsleep i , and Tsetup i denote the aggregate time during one cycle that the system is in the busy state, in the idle state, in sleep i state (i ∈ {0, . . ., n − 1}), and in setup i state (i ∈ {0, . . ., n − 1}), respectively.Thus, Since FIFO is a work-conserving service discipline, we know that In addition, since the idling time timer is reset only in the beginning of a new cycle, we have Tidle = I, and consequently It is also easy to see that Combining these together, we get where, as mentioned earlier, Note that the mean cycle length E[C] is insensitive to the shape of the idling time distribution depending just on its mean value E[I].In addition, for the special case where pik = 1 for some i and k, we clearly have Theorem 1.For any policy belonging to Πmixed, the mean power consumption is given by where E[C] is given in (9).
Proof: By utilizing the theory of regenerative processes, we know that from which (11) easily follows by ( 5)-( 8). 2 Remark: Equation ( 11) applied to the case where I = 0 and setup delays Di are deterministic gives the same result as given in [6,Equation (3)].In addition, for the special case where pik = 1 for some i and k, we clearly have, by (11), , (12) which corresponds to the equation of the mean energy consumption given in [14, Theorem 2].
Theorem 2. For any policy belonging to Πmixed, the mean response time is given by Proof: To determine the mean response time, we continue to utilize the regenerative cycle analysis introduced above.More precisely said, we derive the mean response time E[T ] using a similar approach as Medhi for M/G/1-FIFO vacation models in [15].
Let us start with the mean waiting time E[W ].We will derive it in three parts: 1 • First, due to the FIFO service discipline, the arriving job has to wait until the end of the current service, if the system is busy upon the arrival, and until the end of the service of all the jobs already waiting upon the arrival.Thus, where πbusy denotes the probability that the system is busy upon the arrival, S R is the remaining service time of the job in service, NW is the number of waiting jobs upon the arrival, and the Sj refer to their service times.The standard M/G/1 analysis (applying PASTA and Little's result 2 • In addition, a job that arrives when the server is in one of the states sleep i (i = 0, . . ., n − 1) has to wait until the end of the current sleep state, which is controlled by parameter k, as well as the following setup delay.Thus, where π ik denotes the probability that the current cycle is related to state sleep i and switch-on threshold k and that the arriving job is one of these k jobs.Clearly, we have implying that 3 • Finally, a job that arrives during one of the setup delays Di (i = 0, . . ., n − 1) has to wait until the end of the current setup delay.Thus, where πi denotes the probability that the current cycle is related to state sleep i and that the arriving job is one of the jobs arriving during the corresponding setup delay Di, and D R i refers to the remaining part of the setup delay upon the arrival.Clearly, we have In addition, the standard renewal theory says that . Thus, Formula ( 13) follows now straightforwardly from 1 Remark: Equation ( 13) applied to the case where I = 0 and setup delays Di are deterministic gives the same result as given in [6, Equation ( 1)].In addition, for the special case where pik = 1 for some i and k, we clearly have, by (13), which corresponds to the equation of the mean response time given in [14, Theorem 3]. 3

OPTIMIZATION
In this section, we prove that, for the cost metrics ERWS (1) and generalized ERP (4), the optimal policy is deterministic (choosing exactly one of states sleep i and thresholds k) and the optimal idling time is either I = 0 and I = ∞, which generalizes the earlier results given in [6] and [14] as discussed in Section 1.The result is first split in parts and proved in Propositions 1-4, and finally summarized in Theorem 3.

Optimal idling time distribution
Let us first consider the optimization of the idling time distribution.It follows from ( 9), (11), and ( 13) that both the mean response time E[T ] and the mean power consumption E[P ] are, in fact, insensitive to the shape of the idling time distribution depending just on its mean value E[I] as follows: where constants A1, A2, B1, C > 0 but B2 may be negative.
Proposition 1.For the ERWS cost metric ( 1), the optimal policy in Πmixed has either I = 0 or I = ∞.
Proof: By (15), the objective function ( 1) can be written in the following form: which is clearly a monotonic function of E[I] in the whole interval [0, ∞). 2 Proposition 2. For the generalized ERP cost metric (4), the optimal policy in Πmixed has either I = 0 or I = ∞.
Proof: By (15), the objective function (4) can be written in the following form: If B2 ≥ 0, then the objective function is clearly strictly decreasing so that the optimal idling time is I = ∞.Thus, from this on, we assume that B2 < 0. Let us now consider the function f (x) defined for all real values of x as follows: 1 • First we show that the first derivative f (x) has at most one root.By taking the first derivative of f (x) and rearranging the terms, we have 3 The formula for the mean response time given in [14, Theorem 3] is more complicated but gives (14) after some manipulations when interpreting the symbol σ 2 setup in [14] as the second moment E[D 2  i ] (instead of the variance V[Di]) of the setup delay.
Clearly, the only possible root comes from the last part of this equation, which is a linear function of x.If such a root x0 exists, it satisfies Since B2 < 0, we have f (x0) < 0, which justifies the claim.
From 1 • and 2 • , we deduce that f (x), when restricted to the interval x ∈ [0, ∞), has its minimum value at x = 0 or when x → ∞, which completes the proof. 2

Optimal choice of sleep state and switchon threshold
Now we consider, for the same cost metrics as above, optimization of the distribution p = (pik; i ∈ {0, . . ., n − 1}, k ∈ {1, 2, . ..}), which is used to select the sleep state i and the switch-on threshold k every time when the idling time timer expires.From Propositions 1 and 2, we know that I = 0 or I = ∞ for the optimal policy.On the other hand, if the optimal idling time is I = ∞, the distribution p does not have any role.Thus, we may assume below that I = 0.
It follows from ( 9), (11), and (13) that the mean response time E[T ] and the mean power consumption E[P ] for these three policies can be written as follows: ) where A1, A2, C1, C2, E > 0 and B1 +C1, B2 +C2, D+E > 0 but B1, B2, and D may be negative. 4 4 While we use partly the same symbols, do not confuse these constants with those given in (15).
If D = 0, then an elementary polynomial division results in the following formulas: where Proposition 3.For the ERWS cost metric ( 1), the optimal policy in Πmixed is deterministic, i.e., there are i * and k * such that pi * ,k * = 1.
Proof: 1 • Assume first that D = 0.By ( 16), the objective function ( 1) can be written in the following form: which is a linear and, thus, monotonic function of pik in the interval [0, 1].Thus, any distribution p can be improved by one of the related distributions p 0 or p 1 for any i and k, which proves the claim in this case. 2 • Assume now that D = 0.By (17), the objective function ( 1) can be written in the following form: Dp ik +E , which is clearly a monotonic function of pik in the interval [0, 1].Thus, also in this case, any distribution p can be improved by one of the related distributions p 0 or p 1 for any i and k, which completes the proof. 2 Proposition 4. For the generalized ERP cost metric ( 4), the optimal policy in Πmixed is deterministic, i.e., there are i * and k * such that pi * ,k * = 1.
Proof: 1 • Assume first that D = 0.By (16), the objective function ( 1) can be written in the following form: If B1 and B2 have the same sign, then the objective function is clearly a monotonic function of pik in the interval [0, 1], implying that any distribution p can be improved by one of the related distributions p 0 or p 1 for any i and k.
From this on (until the end of 1 • ), we assume that B1B2 < 0. Consider now the function f (x) defined for x ∈ [0, 1] as follows: By inspecting the first and the second derivatives of f (x), it is a straightforward task to prove that f (x) has its minimum value at x = 0 or x = 1 (cf. the proof of Proposition 2).It follows that again in this case any distribution p can be improved by one of the related distributions p 0 or p 1 for any i and k.
2 • Assume now that D = 0.By (17), the objective function ( 4) can be written in the following form: If C 1 and C 2 have the same sign, then the objective function is clearly a monotonic function of p ik in the interval [0, 1], implying that any distribution p can be improved by one of the related distributions p 0 or p 1 for any i and k.
Thus, from this on, we assume that C 1 C 2 < 0. Let us now consider the function f (x) defined for x ∈ [0, 1] as follows: By again inspecting the first and the second derivatives of f (x), it is easy to prove that f (x) has its minimum value at x = 0 or x = 1 (cf. the proof of Proposition 2).It follows that again in this case any distribution p can be improved by one of the related distributions p 0 or p 1 for any i and k, which completes the proof. 2 Remark: In [6, Theorem (1)] the optimal switch-on threshold for the ERP cost metric is proven to be k * = 1 for exponentially distributed service times.However, we show, via counter-example, that this optimality result does not hold for generally distributed service times by considering job sizes with high variability.
Mean service time of E[S] = 1.0 and arrival rate of λ = 0.5 are assumed giving a load value of ρ = 0.5.The second moment of service time is assumed to be E[S 2 ] = 30.0.Furthermore, we assume that the system is equipped with a single sleep state having the same parameter values as the suspend state given in Section 6 with a deterministic setup delay of E[D] = E[S] = 1.0.With these assumptions, the ERP metric of the standard M/G/1 system, I = ∞, will read as 2560 while that of the sleeping policy with I = 0 is 2329 for k = 1 and 2230 for k = 2. Therefore, the optimal policy, in this case, has a threshold value different from 1.
Intuitively, the reason why the optimal k may be greater than 1 is that with highly variable job sizes it does not hurt the overall delay so much to have k > 1 and wait for some additional jobs to save more energy, because the delays are anyway dominated by the long busy period delays caused by the highly variable job sizes.
All the optimality results proved above are summarized in the following theorem.
Theorem 3.For the cost metrics ERWS (1) and generalized ERP (4), the optimal policy in Πmixed has either I = 0 or I = ∞ and there are i * and k * such that pi * ,k * = 1.
Remark: Below we demonstrate, by constructing a counterexample, that the optimal idling time may be different from I = 0 and I = ∞ if there are multiple terms in the general cost metric (3), unlike as argued in [14].
Assume that new jobs arrive with rate λ = 0.1 and service times are exponential with mean E[S] = 1.0 so that ρ = 0.1.In addition, assume that the only sleep state is state off.The corresponding setup delay D0 is assumed to be deterministic with D0 = 1.0.The power consumption in different states is as follows: Pbusy = Psetup = 1.0,Pidle = 0.6, and Poff = 0.0.With these parameters, we have, for the ordinary M/G/1 queue (i.e., I = ∞), Consider now the general cost metric (3) with parameters w1 = 1, a1 = 1, b1 = 0, w2 = 3, a2 = 0, b2 = 3 so that the objective function is For the ordinary M/G/1 queue with I = ∞, the objective function takes value 1.898, while for the policy with I = 0 and k = 1, it is 2.084.With I = 0 and k > 1, the objective function takes even higher values.On the other hand, it is easy to check that a strictly lower value, 1.776, is achieved when the mean idling time is E[I] = 20.7 and the switch-on threshold k = 1.Thus, the optimal idling time is, indeed, different from I = 0 and I = ∞ in this case.

RESETTING THE IDLE TIME TIMER
Thus far, we have assumed, as in [14], that the idling time timer is reset (to 0) only when it expires.Let us now consider a modified system where the timer is reset after each idle period (whether the timer expired or not).We still assume that I can have a general distribution.
Recall from Section 3 that Tidle refers to the aggregate time during one cycle that the system is in state idle.Now, as the timer is reset after each idle period, we clearly have where I denotes the idling time (as before) and A is an independent exponentially distributed random variable with mean 1/λ.
In the analysis part, the only modification needed is to replace E[I] with E[Tidle] in Equation ( 9) for the mean cycle length E[C] so that With this expression for E[C], the formulas ( 11) and ( 13) for E[P ] and E[T ], respectively, remain still valid.In addition, note that conditions I = 0 and I = ∞ are equivalent with conditions Tidle = 0 and Tidle = ∞, respectively.Thus, using exactly the same arguments as in the optimization part (Section 4), we can conclude that the optimality results in Theorem 3 remain valid even when the timer is reset after each idle period.

NUMERICAL RESULTS
From the analytical results we already know that there is no need to randomize between sleep states.However, the  optimal sleep state under specific system parameters can only be determined numerically.To address this question we study three energy states that are available in modern servers.These are the common sleep states suspend and hibernate along with the off state, which shuts down the server completely.In the suspend state, operational state of the server is saved in the RAM so as to take shorter time to set it up.The hibernate state writes the system state in a hard disk and turns all devices off consuming less power than the suspend state at a price of higher setup delay.
Power consumption values of Pbusy = Psetup = 200W, Pidle = 120W Ps = 15W, Ph = 5W and Poff = 0 will be used in this demonstration with subscripts s and h representing the suspend and hibernate states, respectively.Moreover, mean setup delays of 100 s, 50 s and 10 s will be used for the off, hibernate and suspend states, respectively.All values are taken from experimental measurements performed in [7,10] while the service times obey an exponential distribution, where the mean is exaggerated to E[S] = 1 s for illustration.Weighting factors of w1 = 1 and w2 = 0.75 are used for all the ERWS plots.Using these parameters, the system cost of the sleeping policies relative to the non energy-aware M/G/1 system is illustrated in Figures 1-3.Thus, whenever a policy has an ERP or ERWS score of less than 1, it has a lower cost than the standard M/G/1 system.
The normalized ERP and ERWS metrics under light load are depicted in Figure 1 as a function of the switch-on threshold.As already proven in [6, Theorem (1)] for exponentially distributed service times, the upper panel shows that k = 1 is the optimal choice that minimizes ERP.In the case of ERWS, the lower panel of Figure 1 clearly shows that the optimal value of k may lie elsewhere.In line with the findings in [10] suspend gives the best ERP and ERWS amongst the sleep options.This is because setup delay from the suspend sleep state is significantly lower while the power consumption is still kept reasonably low.This affects ERP the most since it gives equal weight to both performance and power reductions.
Figure 2 shows the normalized ERP and ERWS metrics as a function of system load.The ERWS curves in the lower panel are produced by first optimizing with respect to the switch-on threshold at a given load value.With the suspend state giving the lowest cost, all the sleeping policies are found to have a worse ERP as compared to the standard M/G/1 system.However, suspend is found to have significant improvement in ERWS for up to moderate system load.Motivated by this observation, we studied the ERP of suspend further.If technological advancements were to reduce the setup delay to the level of the mean service time, the system would have had the ERP curve given in Figure 3.In this ideal case, a significant reduction in system cost can be achieved by suspending the server whenever the load is low.

CONCLUSIONS
We analyzed the energy-performance tradeoff in the M/G/1 queue, where the server can utilize sleep states to reduce the energy consumption.Specifically, we considered policies, where first, after becoming idle, the server waits if the idling time timer expires.If it does, the server goes into sleep state, which may be selected randomly as well as the number of jobs needed in the queue before restarting the server again.However, the cost of reduced energy consumption is the extra delay, the setup delay, needed to get the server operational when waking up from the sleep state.In our model, the service time, the setup delay and the idling time timer may have a general distribution.This model includes as special cases certain previously studied models.
The mean performance and power metrics were derived using standard regenerative techniques.Our main result for ERWS and ERP cost metrics provided the characterization of the optimal policy: either the server never uses any of the sleep states, i.e., only uses the idle state, or the server directly enters some specific sleep state without any idling time.Moreover, the optimal threshold for the the number of jobs in the sleep state was shown to be deterministic.However, we also demonstrated that these results may not hold for more general metrics.Our numerical results highlighted that with realistic values for the power consumption and setup the potential gain from using sleep states may be limited, except at light loads.

Figure 1 :
Figure 1: ERP and ERWS of the sleeping policies as a function of the switch-on threshold, normalized with respect to the non energy-aware M/G/1 system.

Figure 2 :
Figure2: ERP and ERWS of the sleeping policies as a function of system load, normalized with respect to the non energy-aware M/G/1 system.At a given load value, each policy is optimized with respect to the switch-on threshold to produce these plots.

2 •
Now we show that any root of f (x) is a local maximum point.Let us assume that such a root, x0, exists.The second derivative test can be applied to prove this claim since f (x) is twice differentiable at x0.By taking the second derivative of f (x) and applying it at x0, we get