Flow coupling and stochastic ordering of throughputs in linear networks

Robust estimates for the performance of complicated queueing networks can be obtained by showing that the number of jobs in the network is stochastically comparable to a simpler, analytically tractable reference network. Classical coupling results on stochastic ordering of network populations require strong monotonicity assumptions which are often violated in practice. However, in most real-world applications we care more about what goes through a network than what sits inside it. This paper describes a new approach for ordering flows instead of populations by augmenting network states with their associated flow counting processes and deriving Markov couplings of the augmented state–flow processes.


INTRODUCTION 1.1 Stochastic ordering of network flows
Robust estimates for the performance of a complicated queueing network can be obtained by showing that the process X describing the number of jobs in the network is stochastically comparable to a simpler, analytically tractable reference network X .Classical coupling results on stochastic ordering of X and X require strong monotonicity assumptions [5,7,9,10,15] which are often violated in practice.However, in most real-world applications we care more about what goes through a network than what sits inside it.This paper describes a new alternative approach for ordering flows instead of populations by augmenting the network states X and X with their associated flow-counting processes F and F and deriving Markov couplings of the augmented state-flow processes (X, F ) and (X , F ) in an extended state space.
Earlier methods applicable for ordering of flows are mostly based on Markov reward comparison techniques (e.g.[1,3,12,13,14]).While more limited in scope than the general Markov reward approach, the flow coupling technique presented here, when applicable, yields stronger ordering results using simpler analysis.This paper will demonstrate this in the context of open linear queueing networks with general state-dependent arrival and service rates.

Motivating example
Consider a network of two queues in series where queue 1 and queue 2 have buffer capacities s1 and s2, respectively.Jobs arriving while queue 1 is full are rejected and lost, and the server of queue 1 halts when queue 2 is full.When all interarrival times and job sizes are independent and exponentially distributed, the network population can be represented as a Markov jump process X orig in the state space where β is the arrival rate of offered jobs, δi(xi) is the service rate at queue i when queue i has size xi, ei is the i-th unit vector in Z 2 , and 1(A) is the indicator function which returns one if statement A is true and zero otherwise.For example, when δi(xi) = cixi, the system corresponds to a multi-server queue where all servers operate at rate ci.When the system is irreducible, the long-run mean loss rate and other equilibrium statistical characteristics can in in principle be evaluated by solving a linear equation for the equilibrium distribution.However, because the system is not reversible, solving the linear problem analytically or numerically is hard for large s1, s2.
To obtain a computationally tractable upper bound for the equilibrium loss rate, van Dijk and van der Wal [13] introduced a modification of the network dynamics so that arrivals are blocked also in states where queue 2 is full, and the second server halts when queue 1 is full.This so-called balanced system has a product-form equilibrium distribution and can be represented as a Markov jump process X alt with transitions x − e2 at rate δ2(x2)1(x1 < s1). ( Because the state (s1, s2) is transient for X alt , it is natural to define X alt on the space S alt = S orig \ {(s1, s2)}.
Earlier methods applicable for ordering of flows are mostly based on Markov reward comparison techniques (e.g.[1,3,12,13,14]).While more limited in scope than the general The balanced service system X alt employs a stricter admission policy and provides less service for queue 2. Hence it is intuitively feasible to assume that the counting processes F orig in (t) and F alt in (t) describing the number of accepted jobs up to time t are ordered according to with respect to a suitable stochastic order.However, we are faced with the following conceptual problem: If X alt accepts less jobs, it should have shorter queues, which should imply that X alt spends less time in blocking states, and hence X alt should accept more jobs.For this reason, a simple sample path argument cannot be used to prove (3).
Under the natural assumption that δ1(x1) and δ2(x2) are increasing1 functions, Van Dijk and van der Wal [13] proved that by uniformizing the Markov processes into a discrete-time chain and applying inductive Markov reward comparison techniques.They also argued that a simpler coupling proof is not possible due to nonmonotone effects caused by the blocking phenomena.
In this paper it will be shown that although neither of the above Markov processes is monotone in the strong coordinatewise stochastic order, a strong coupling argument for proving (3) is nevertheless possible.To accomplish this, this paper will introduce a Markov coupling in an extended space which carries redundant information about flow counting processes associated with the network population.As an application we obtain a simple proof of (3) in the strong stochastic sense which at the same time strengthens (4) and greatly simplifies its lengthy proof given in [13].

NETWORK POPULATION PROCESSES 2.1 Markov dynamics
Consider a network consisting of a finite set of nodes N = {1, . . ., n} where jobs randomly move across directed links L ⊂ (N ∪ {0}) 2 , and where node 0 represents the outside world.The state of the system is denoted by x = (x1, . . ., xn) ∈ S where S ⊂ Z N + and Z+ denotes the positive integers.The network dynamics is modeled as a Markov jump process X = (X1(t), . . ., Xn(t)) t≥0 in state space S with transitions where ei denotes the i-th unit vector in Z n , and e0 stands as a synonym for zero.Here • Xi(t) is the number of jobs in node i at time t, • αi,j (x) for i, j ∈ N is the instantaneous transition rate of jobs from node i to node j at state x = (x1, . . ., xn), • α0,i(x) and αi,0(x) are the arrival and departure rates of jobs for node i at state x = (x1, . . ., xn).
A collection of transition rates αi,j : S → R+ and an initial state X(0) defines such a Markov jump process in S, when αi,j (x) = 0 for all x ∈ S such that x − ei + ej ∈ S and satisfy the standard regularity condition which guarantees that the Markov jump process is nonexplosive (see e.g.[2]).

Augmented state-flow process
The state-flow process associated to population process X generated by transition rates αi,j is a Markov jump process (X, F ) taking values in S × Z L + and having transitions (x, f ) → (x − ei + ej, f + ei,j) at rate αi,j (x), (i, j) ∈ L, where ei,j denotes the unit vector in Z L + having its (i, j)coordinate equal to one and other coordinates zero.Here • Xi(t) is the number of jobs at node i at time t This process is redundant in that the second component of (X, F ) may be recovered from F (0) and the path of X by the formula where X(s−) denotes the left limit of X at time s.Adding this redundancy allows to derive useful non-Markov couplings of X in terms of Markov couplings of (X, F ), as we shall soon see.

Strong stochastic order
Let us recall some standard notations and facts about strong stochastic ordering of random processes.For random vectors A and B in R n , we denote A ≤st B and say that A is less than B in the strong stochastic order if E φ(A) ≤ E φ(B) for all φ : R n → R which are increasing with respect to the coordinatewise order on R n and for which the expectations are defined.

Coupling of population processes
Fix a network with nodes N = {1, . . ., n} and directed links L ⊂ (N ∪ {0}) 2 , and consider two population processes X in S ⊂ Z N + and X in S ⊂ Z N + , generated by state-dependent transition rates αi,j (x) and α i,j (x), respectively.In most applications the state spaces S and S are assumed to be identical, but this restriction is not needed for the results developed in this paper.
A natural and simple way to couple two Markov population processes of the above type is to force both processes to locally take identical steps with as high rate as possible.This so-called marching soldiers coupling [4] of the Markov jump processes X and X is defined as a Markov jump process ( X, X ) in S × S having the transitions Here we use the shorthands a∧a = min{a, a } and a+ = max{a, 0}.By inspecting the marginal transition rates for each state x ∈ S and x ∈ S , one can check that the process X (resp.X ) is a Markov jump process by itself and has the same transition rates as X (resp.X ).That is, ( X, X ) is a Markov coupling of X and X .

Coupling of state-flow processes
The marching soldiers coupling of state-flow processes (X, F ) and (X , F ) associated to population processes X and X is defined analogously as a Markov jump process ( X, F , X , F ) in the state space (S ×Z L + )×(S ×Z L + ) having the transitions for (i, j) ∈ L, where Ti,j (x, f ) = (x −ei +ej, f +ei,j) denotes the extended state obtained from state (x, f ) after moving one job from node i to node j.

Flow balance
If (X, F ) is the state-flow process associated to a population process X, then for all t ≥ 0. This flow conservation equality shows that Xi(t) − j:(j,i)∈L Fj,i(t) + j:(i,j)∈L Fi,j (t) remains constant over time for all nodes i.As a consequence, any coupling ( X, F , X , F ) of state-flow processes (X, F ) and (X , F ) automatically preserves the relation for all nodes i, in the sense that the set of pairs (x, f ) and (x , f ) related according to ( 5) is absorbing for ( X, F , X , F ).
In queueing terminology, λ(x) is the arrival rate of jobs to node 1, and μi(x) can be thought of as the aggregate service rate at node i.The fact that λ(x) and μi(x) may depend on the full network state x = (x1, . . ., xn) allows to model queueing systems with admission control in front, and blocking and interference effects among the servers.We assume that λ(x) = 0 for all x ∈ S such that x + e1 / ∈ S and μi(x) = 0 for all x ∈ S such that x − ei + ei+1 / ∈ S.

Strong ordering of flows
Consider now two population processes X and X on the open linear network of n nodes generated by state-dependent transition rates (λ, μ1, . . ., μn) and (λ , μ 1 , . . ., μ n ), respectively.Here the state spaces S of X and S of X are subsets of Z N + which in most applications are identical, but for the following result this restriction is not needed.
The following is the main result of this paper.It gives a sufficient condition for the strong stochastic ordering of flow counting processes associated to X and X .For convenience, we define i + 1 := 0 for i = n below.
Proof.It is sufficient to construct a coupling of the stateflow processes (X, F ) and (X , F ) for which (10) holds for all i and all t ≥ 0 with probability one.Let ( X, F , X , F ) be a marching soldiers coupling of (X, F ) and (X , F ) started at the (possibly random) initial state (X(0), F (0), X(0), F (0)), as defined in Sec.2.4.By assumption, F (0) = F (0) is the zero vector in Z L + .Note that the above vector couples the initial states of (X, F ) and (X , F ) because X(0) and X (0) have the same distribution.The latter assumption also implies that X(0) takes its values in S∩S almost surely.

We define a relation between state-flow pairs
for all i = 0, 1, . . ., n and if (5) holds for all i = 1, . . ., n.We note that ( X(0), F (0)) ∼ ( X (0), F (0)) almost surely.To finish the proof it suffices to show that the set of state-flow pairs that are related according to ∼ is an absorbing set for the marching soldiers coupling.Note that both sides of (5) are invariant to any possible transition of the processes.Hence we only need to show that none of the inequalities (11) can ever be broken by any transition of the coupled process.
Let us next show that (11) cannot be broken for 1 ≤ i ≤ n − 1.Consider a state-flow pair related according to (x, f ) ∼ (x , f ).Again, we only need to study the case where fi,i+1 = f i,i+1 .Then the flow conservation equality (5) at node i implies that for node i + 1 implies that Thus xi ≤ x i and xi+1 ≥ x i+1 which in light of (8) imply that μi(x) ≤ μ i (x ).This shows that the marching soldiers coupling has zero rate for the transition ((x, f ), (x , f )) → ((x − ei + ei+1, f + ei,i+1), (x , f )).But this is the only transition which potentially could break (11) for i.
Let us finally show that (11) cannot be broken for i = n.Consider a state-flow pair related according to (x, f ) ∼ (x , f ).Again, we only need to consider next that case where fn,0 = f n,0 .Then the flow conservation equality (5) at node n implies that Thus xn ≤ x n which in light of ( 9) implies that μn(x) ≤ μ n (x ).This shows that the marching soldiers coupling has zero transition rate for the transition ((x, f ), (x , f )) → ((x + en, f + en,0), (x , f )).But this is the only transition which potentially could break (11) for i.
Because the marching soldiers coupling may never exit the set of ordered state-flow pairs, we conclude that ( X(t), F (t)) ∼ ( X (t), F (t)) for all t ≥ 0, and especially Fi,i+1(t) ≤ F i,i+1 (t) for all i and all t ≥ 0 almost surely.

Strong ordering of populations
To understand how Theorem 1 is structurally different from more well-known ordering and coupling results for Markov population processes, let X and X as in the previous section.The following result gives a sufficient condition for the strong stochastic ordering of the population processes X and X with respect to the coordinatewise order on R n .For vectors in R n we write (x1, . . ., xn) ≤ (x 1 , . . ., x n ) if xi ≤ x i for all i.For convenience, we define i + 1 := 0 for i = n below.
Let ( X, X ) be a marching soldiers coupling of X and X as described in Sec.2.4.1, started at the initial state ( X(0), X (0)).We will show that the marching soldiers coupling never exits the set of state pairs ordered according to the coordinatewise order, that is, the set {(x, x ) ∈ S × S : x ≤ x } is absorbing for the Markov process ( X, X ).
Consider a pair of states such that x ≤ x , and let us try to break the ordering x1 ≤ x 1 .This is possible in a single transition only if x1 = x 1 , in which case (12) implies that λ(x) ≤ λ (x ) and μ1(x) ≥ μ 1 (x ).But then the transitions (x, x ) → (x+e1, x ) and (x, x ) → (x, x −e1) both have zero rate for the marching soldiers coupling.These are the only transitions for the marching coupling which could break the relation x1 ≤ x 1 .
Consider next a pair of states such that x ≤ x , and let us try to break the ordering xi ≤ x i for some i ≥ 2. This is possible in a single transition only if xi = x i , in which case (13) implies that μi−1(x) ≤ μ i−1 (x ) and μi(x) ≥ μ i (x ).But then the transitions (x, x ) → (x − ei−1 + ei, x ) and (x, x ) → (x, x − ei + ei+1) both have zero rate for the marching soldiers coupling.These are the only transitions for the marching coupling which could break the relation xi ≤ x i .
We conclude that X(t) ≤ X (t) for all t ≥ 0 almost surely, and therefore the claim follows.
Note that Theorem 2 can also be proved as a consequence of a generic relation preservation result in [7, Example 5.7] (see alternatively [8]), or by applying the transition rate conditions in [9,15].

APPLICATION: THROUGHPUT ORDER-ING IN A TANDEM QUEUE
Let us now revisit the tandem queueing system of Sec.1.2.The balanced model described by ( 2) corresponds to a population process X on a 2-node linear network where S = S alt and In this case Theorem 2 cannot be applied to order populations according to X(t) ≤st X (t) because condition (13) fails for i = 2 due to μ2(x) < μ 2 (x ) when x1 = x 1 = s1 and 0 < x2 = x 2 < s2.Neither can Theorem 2 cannot be applied to order populations according to X (t) ≤st X(t) because condition (12) fails due to λ (x ) > λ(x) when x 1 = x1 < s1 and x 2 < x2 = s2.
Nevertheless, the augmented state-flow process can be coupled with the help of Theorem 1. Indeed, the conditions of Theorem 1 are valid if and only if the service rates δ1(x1) and δ2(x2) are increasing.When this natural monotonicity condition is satisfied and when both systems are started at the same initial state and zero flow counters, Theorem 1 implies that all flow counting processes are ordered by F0,1(t) ≤st F 0,1 (t), F1,2(t) ≤st F 1,2 (t), F2,0(t) ≤st F 2,0 (t).
Because F0,1(t) and F 0,1 (t) are the counting processes of accepted jobs in the balanced system and the original system, we obtain (3).

CONCLUSIONS
This paper discussed the strong stochastic ordering and coupling of network populations and their flow counting processes.Easily verifiable sufficient conditions were given for the transition rates of population processes on open linear networks which imply that the associated flow counting processes can be ordered using a natural coupling in the augmented space of state-flow processes.Important open problems include (i) to study into what extent the given sufficient conditions are also necessary and (ii) to extend the analysis into networks with two-way flows and more general network topologies.These problems are subjects of ongoing research.
[7, admit a coupling which is ordered in the sense that Ât ≤ Bt for all t almost surely, then (At) ≤st (Bt).As a consequence of Strassen's coupling theorem, the converse implication is also true whenever the paths of (At) and (Bt) are rightcontinuous with left limits (e.g.[7, Thm 4.6]).
[10,11]l-valued random processes (At) and (Bt) indexed by a time parameter t we denote (At) ≤st (Bt) if (At 1 , ..., At n ) ≤st (Bt 1 , ..., Bt n ) for all finite collections of time parameters (t1, ..., tn).Strong stochastic order allows to compare means in the sense that At ≤st Bt implies E At ≤ E Bt whenever At and Bt are positive or have finite means.Perhaps more importantly, it also allows to compare upper tail events in that At ≤st Bt always implies P(At > s) ≤ P(Bt > s) for all real numbers s.In fact the latter property is equivalent to At ≤st Bt, see e.g.[10,11].For random vectors A and B we denote A =st B if A and B have the same distribution.This definition is extended to random processes by denoting (At) =st (Bt) if (At 1 , . .., At n ) =st (Bt 1 , . .., Bt n ) for all finite collections of time parameters (t1, . .., tn).A bivariate random process ( Ât, Bt) indexed by time parameter t is a coupling of random processes (At) and (Bt) if ( Ât) =st (At) and ( Bt) =st (Bt).A simple computation using the definitions shows that if (At) and (