Cooperative MIMO Relaying with Orthogonal Space-Time Block Codes in Wireless Channels with and without Keyholes ★

Cooperative multiple-input multiple-output (MIMO) relaying is investigated in the paper. We introduce DFAF selection MIMO relaying, where the relay equipped with multiple antennas can adaptively switch between decode-and-forward (DF) and amplify-and-forward (AF) according to its decoding state of the sourcemessage. We consider two wireless environment scenarios: 1)The scenario with traditional channels are considered firstly. We analyze the outage performance of DF-AF selection MIMO relaying, and a closed-form expression is derived. In addition, the diversity order is obtained based on the expression. For comparison purpose, we also obtain the closed-form outage probability and the diversity order for the AF MIMO relaying and the DF MIMO relaying. 2)We investigate the cooperative MIMO relaying in the presnece of keyholes secondly. We present performance analysis of orthogonal space-time block coded transmission for a cooperative MIMO relaying system with keyholes. For DF MIMO relaying, exact outage probability and symbol error probability (SEP) are obtained. Regarding AF MIMO relaying and DF-AF selection MIMO relaying, the lower and upper bounds are derived. In both traditional and keyhole scenarios, theoretical analysis which has been further verified through Monte-Carlo simulations demonstrate that the DF-AF selection MIMO relaying has better performance than the AF MIMO relaying and the DF MIMO relaying. Received on 09 October 2013; accepted on 09 December 2013; published on 14 July 2014


Introduction
Multiple-input multiple-output (MIMO) techniques have gained huge attention in the past decade because of their high spectral efficiency in both single-user and multi-user communications [2,3].Deploying multiple antennas at each node is a promising approach to solve the increasing demand for data-rate-intensive a virtual antenna array to achieve spatial diversity.Several cooperative diversity protocols including amplifyand-forward (AF), decode-and-forward (DF), selection relaying and incremental relaying, were discussed in [6].DF-AF selection relaying protocol, where each relay can adaptively switch between DF and AF according to its local SNR, has been developed and investigated in [7]- [10].
More recently, MIMO relaying technologies that exploit the cooperative diversity as well as the advantages of MIMO systems by accommodating multiple antennas at the relay nodes have been well developed [11,12].In [13], the authors obtained the bounds for the capacity of MIMO relay channels.A DF cooperative MIMO relay channel with orthogonal space-time block codes (OSTBC) was analyzed in [14].MIMO cooperative diversity with scalar-gain AF relaying was studied in [15].MIMO with noncoherent AF relaying was considered in [16] and [17].In [18], the authors presented performance analysis of a AF cooperative MIMO relaying system based on Alamouti scheme [19].The MIMO relay channels with the channel-state information (CSI)-assisted AF relaying technique was considered in [20].DF MIMO relay channels employing Maximum Likelihood (ML) detection in Rayleigh fading was investigated in [21].MIMO relaying with AF for UWB ad hoc networks was analyzed in [22].In [23], DF relaying for MIMO ad hoc networks was studied, the authors demonstrated that the use of cooperative relay in a MIMO framework could bring in a significant throughput improvement.
It has been shown, both theoretically and experimentally, that degenerate channel phenomena termed "keyholes" may exist under realistic assumptions [24] - [27].A spatial MIMO keyhole is a propagation scenario where the channel gain matrix has only unit rank, even when multiple uncorrelated antennas are employed.Thus, keyhole will degrade the MIMO channel capacity to that of a single-input single-output (SISO) channel.
To date, the effect of keyholes on performance of STBC over MIMO channels has been well investigated.The average symbol error rate (SER) of orthogonal space-time code (OSTBC) [28] with M-ary phase shift keying (M-PSK) and M-ray quadrature amplitude modulation (M-QAM) constellations over keyhole MIMO channels was analyzed in [29].In [30], the authors derived exact analytical closed-form expressions for the ergodic capacity and information outage probability of keyhole MIMO channels in Nakagami-m fading environments.Exact expressions for the SER of OSTBC over a spatially correlated MIMO channel, in which the signal propagation suffers from a keyhole effect was derived in [31].The performance of OSTBC in MIMO fading channels under keyhole condition was analyzed in [32].The SER and BER of OSTBC with antenna selection over keyhole fading

Table 1. Summary of related papers
Typical Papers MIMO relaying [11] - [23] MIMO with keyholes [29] - [40] MIMO Relaying with keyholes [41] - [44] channels were examined in [33].Exact expressions of SER of OSTBC in Nakagami-m keyhole channels with arbitrary fading parameters and the closed-form asymptotic expressions were derived in [34].SER/BER and outage probability of OSTBC with M-PSK and M-QAM in keyhole MIMO fading channels were studied in [35].Pairwise error probability (PEP) analysis of general space-time codes (STCs) in keyhole conditions was presented in [36].In [37], the asymptotic PEP of STCs in generalized keyhole fading was obtained.In addition, the keyhole can be viewed as a special case of double-scattering [38].In [38], analytical performance of Rayleigh-product MIMO channels (a special case of double scattering MIMO channels) was studied.Furthermore, the keyhole channel, which is regarded as a special case, was investigated.With respect to Rayleigh-product MIMO channels, the diversitymultiplexing tradeoff (DMT) analysis and the ergodic sum rate analysis can be found in [39] and [40], respectively.
There are a few works on keyhole MIMO relay channels.In [41], the authors studied MIMO relay channels in the presence of keyhole effect.The ergodic capacity is investigated when the source-relay channel is keyhole-free.Moreover, they demonstrated that cooperative diversity can mitigate keyhole effects.Hence it is important to study the cooperative MIMO relay channels with keyholes.In [42], the authors derived the exact ergodic capacity for MIMO AF relaying systems with a multi-keyhole effect on the relay-destination channel.In [43], the authors investigated the performance of MIMO AF relay networks with keyhole and spatial correlation.The SEP and outage probability were analyzed in Rayleigh fading environments when the source-relay link is keyhole-free.
Previous works related are presented in Table 1.
In this paper, we consider the cooperative MIMO relaying in the absence and presence of keyholes, respectively.First, DF-AF selection MIMO relaying is introduced in traditional (keyhole free) wireless channels.We investigate the outage probability of a cooperative DF-AF selection MIMO relaying system with OSTBC and selection diversity.A closed-form solution at arbitrary SNR is obtained and the diversity order is obtained based on the expression.Next, we investigate the MIMO relaying in the presence of (ii) We investigate MIMO relaying in the scenario that the keyholes exist.
• We consider the MIMO relay channels when the source-destination, the sourcerelay and the relay-destination channels all incur keyhole effect in this paper.As it makes assumptions on the mobility pattern or location of neither the relay nor the destination, this scenario is more practical and challenging.
• The outage probability and SEP of STBC over keyhole channels for AF MIMO relaying in Nakagami-m fading environments are investigated.Furthermore, the outage probability and SEP of keyhole DF MIMO relaying system and keyhole DF-AF selection MIMO relaying channels are considered.
(iii) We compare the performance of the three MIMO relaying schemes (i.e., the DF MIMO relaying, the AF MIMO relaying and the DF-AF selection MIMO relaying) in the scenarios with and without keyholes, respectively.We find that DF-AF selection MIMO relaying has the best performance in both scenarios.
Throughout this paper, the notations in Table 2 will be used.

System model
We consider a cooperative MIMO communication system as depicted in Figure 1, where the source, relay, and destination terminals have n s , n r and n d antennas, It is assumed that the instantaneous CSI is available at the receiver, i.e., F is available at the relay and the destination knows F, H, and G.We assume that the source and the relay can employ OSTBC encoding.A half-duplex relaying protocol where each transmission period is divided into two time slots is assumed.In the first time slot, the source transmits OSTBC coded signal to the destination as well as the relay.In the second time slot, the relay processes the received signal and forwards the processed signal to the destination according to some specific relaying protocol.In DF MIMO relaying, the relay transmits the decoded signal using OSTBC when the source message can be correctly decoded.Otherwise, the relay remains idle.In AF MIMO relaying, the relay simply amplifies and forwards the received signal.In this paper, we introduce an efficient MIMO relaying scheme referred to as DF-AF selection MIMO relaying.In the DF-AF selection MIMO relaying scheme, the relay equipped with multiple antennas could adaptively switch between DF protocol and AF protocol.Specifically, if the relay could fully decode the source message, it decodes the source message, and re-encodes the received signal by using OSTBC before forwarding the signal to the destination.Otherwise it amplifies and forwards the received signal to the destination.Maximum Likelihood (ML) detection is used at all receivers in the two time slots.The destination uses a certain combining technique to combine the signals of two time slots coming from the source and the relay to decode the information.

MIMO relaying without keyhole
In this section, we analyze the introduced DF-AF selection MIMO relaying in the cases that no keyhole exits.We assume that all channel matrices are assumed to undergo independent Rayleigh fading with elements obeying CN (0, 2).As outage probability is an important performance measure that is commonly used to characterize a wireless communication system, we focus  The cumulative density function (c.d.f.) of a random variable X f X (x) The probability density function (p.d.f.) of a random variable X E X (•) The expectation operator associated with X. Specially, X = E X (X).Ψ X (s) The moment generating function (m.g.f.) associated with a random variable X, which is defined by The gamma function The incomplete gamma function The v th order modified Bessel function of the second kind The Whittaker function An random variable X has the p.d.f.given by f X (y

||M|| F
The Frobenius norm of a matrix M CN (µ, σ 2 ) Circularly symmetric complex Gaussian distribution with mean µ and covariance σ 2 C n The set of n × 1 complex vectors on the outage performance analysis.The closed-form outage probability is obtained, and the diversity order is derived thereafter.Since combining techniques do not affect the diversity order, we utilize selection combining (SC) for simplicity and conciseness in this scenario.In addition, we perform comparisons with the DF MIMO relaying and the AF MIMO relaying.

Outage probability of DF-AF selection MIMO relaying
A closed-form expression of outage probability valid at arbitrary SNR is obtained in the following theorem.
Theorem 1.The outage probability of the DF-AF selection MIMO relaying scheme P out can be expressed as where α 0 = Rn s N 0 2c 0 P 0 , α 1 = Rn s N 0 2c 1 P 0 , and α 2 = Rn r N 0 2c 2 P 1 .R is the rate of the OSTBC.1 ∆ = γ th = 2 2R − 1. c 0 , c 1 and c 2 represent the distance dependent power transfer factors for the source-destination, source-relay and relaydestination channels respectively.P 0 , P 1 denote the transmit power of the source and the relay respectively.N 0 is the variance of the Gaussian noise at each receive antenna.
Proof.The equivalent instantaneous SNR per symbol of source-destination, source-relay and relay-destination channels are [45].From the assumption of the channel matrix, it can be derived that First, it can be derived that c.d.f. of Y ∼ G(m) can be given by Consequently, the instantaneous equivalent end-toend SNR per symbol at the destination is where ξ denotes the decoding state at the relay, 2 and and The outage probability can be given by 2 Approximately, if the source-relay link is able to support a given transmission rate R, i.e., By conditional probability and the Theorem of Total Probability, (6) can be rewritten as (a) holds since when ξ = 0, i.e., γ 1 < ∆, we have 2) and ( 12) along with some simple manipulations, we arrive at (1), which completes the proof.Theorem 2. The diversity order of the DF-AF selection MIMO relaying scheme is given by

Diversity analysis of DF-AF selection MIMO relaying
Proof.The lower gamma function as b → 0 [46], where ≃ denotes asymptotic equality.Let , and ∼ SNR −n r n s .Consequently, we obtain i.e., the diversity order is n d n s + n r min{n s , n d }.
Remark: If n s < n d , the diversity order is n s (n d + n r ).Otherwise, the diversity order is n d (n s + n r ).

Comparison of DF-AF selection MIMO relaying with AF MIMO relaying and DF MIMO relaying
First, we give the closed-form expressions of the outage probability for AF MIMO relaying and DF MIMO relaying.
Theorem 3. The outage probability of AF MIMO relaying is given by (10).

Remark: The DF-AF selection MIMO relaying and the AF MIMO relaying have the same diversity order.
Theorem 5.The outage probability of DF MIMO relaying is given by Proof.For DF MIMO relaying, the instantaneous equivalent end-to-end SNR can be expressed as γ df = max (γ 0 , ξγ 2 ) .Therefore, the outage probability can be obtained by Combining ( 4), ( 5), (2), and ( 19), ( 17) can be derived.
Remark: The outage probability of the DF-AF selection MIMO relaying and that of the DF MIMO relaying are the same when SC is utilized, and the diversity order is the same thereafter.
When MRC is used, +1 .Thus, the outage probability of DF-AF selection MIMO relaying is less than that of DF MIMO relaying and that of AF MIMO relaying, i.e., P out < P af , P out < P df .

MIMO relaying in the presence of keyholes
Keyhole effect (as illustrated in Fig. 2), under which a MIMO channel has uncorrelated spatial fading between antenna arrays but a rank-deficient transfer matrix, may exist in MIMO fading environments in realistic propagation environments.Keyhole effect will lead to significant performance degradations.Fortunately, recent researches demonstrate that cooperative diversity can mitigate keyhole effects [41].Then we investigate the cooperative MIMO relaying in the keyhole scenario.
In this section, we consider the scenario that all MIMO channels incur keyholes .Due to the keyhole effects, We assume independent Nakagami-m fading on both sides of the keyhole.Elements of h 1 , h 2 , f 1 , f 2 , g 1 , and g 2 are statistically independent.The magnitudes of elements of h 1 , h 2 , f 1 , f 2 , g 1 , and g 2 are modeled as Nakagami-m variants with general fading parameters m h 1,i , m h 2,j , m f 1,k , m f 2,l , m g 1,m , and m g 2,n whereas the corresponding phases are uniformly distributed in [0; 2π).The destination uses Maximal Ratio Combining (MRC) to combine the signals of two time slots coming from the source and the relay.The outage probability and SEP of OSTBC over MIMO relay channel with keyholes are analyzed in this section.First, some m.g.f.s are given as preliminary preparation.Next, m.g.f.based method for computing the outage probability and SEP is introduced.Then we investigate the outage probability and SEP for the DF MIMO relaying, the AF MIMO relaying and the DF-AF selection MIMO relaying over keyhole channels.Furthermore, by comparison, we derive that DF-AF selection MIMO relaying has the best performance. Denote , and , where R is the rate of the OSTBC, P 0 , P 1 represent the transmit power of the source and the relay. and respectively [45,49].

M.g.f.-based method
Let γ denote the total instantaneous received SNR.For M-PSK, the outage probability can be computed as [50] where γ th = 2 (K+1)R − 1.Meanwhile, SEP can be computed as where g psk = sin 2 π M .Remark: f out () and f sep () are mappings from a function space to [0, 1].

Performance analysis
In this subsection, we first consider the DF MIMO relaying, the AF MIMO relaying, and the DF-AF selection MIMO relaying over keyhole channels respectively.Subsequently, we compare the three protocols.
The exact outage probability and SEP of DF MIMO relaying over keyhole channels is given by the following theorem.Theorem 6.The outage probability and SEP of DF MIMO relaying over keyhole channels can be given by and where P e = f sep Ψ γ 1 (s) , Ψ γ 0 (s), Ψ γ 1 (s), and Ψ γ 2 (s) can be given by Lemma 1.
Proof.By ( 28), the symbol error probability at the relay over source-relay channel is given by When MRC is used at the destination, the instantaneous total SNR at the destination is where I denotes the decoding state, I = 1 if the relay could correctly decode, else I = 0, i.e., Pr{I = 0} = P e and Pr{I = 1} = 1 − P e . 3So we can derive f I γ 2 (x) = (1 − P e )f γ 2 (x) + P e δ(0).Consequently, we get Considering the assumption of dependency, it yields that Substituting ( 31) and ( 33) into (34) and making some manipulation, we have Then, substituting ( 35) into ( 27) and ( 28), we arrive at (29) and (30).
Using ( 29), (30) and Lemma 1, we can obtain the exact expressions for outage and SEP.This will be useful in numerical evaluation for performance of DF keyhole MIMO relay channels.
The following theorem gives the lower and upper bounds on the outage probability and SEP of AF MIMO relaying over keyhole channels. 3Correctly decode or not is exactly accessed here and thereafter.
Proof.When MRC is used at the destination, the instantaneous total SNR at the destination is On the other hand, when γ 1 + γ 2 ≫ 1, i.e., high SNR, we have Using the dependency assumption and Proposition 3 in Appendix, we have and Consequently, the outage probability and SEP for AF protocol can be bounded by and Substituting ( 40) and ( 41) into ( 42) and ( 43), ( 36) and ( 37) can be obtained.
It is not difficult to see that when |γ 1 − γ 2 | is sufficiently large, the lower bound will become tight, i.e., P out ≈ f out Ψ γ 0 (s)Ψ min{γ 1 ,γ 2 } (s) , When |γ 1 − γ 2 | is sufficiently small and γ 1 is sufficiently large, the upper bound will become tight.Then we have s .The following theorem gives the lower and upper bounds for the outage probability and SEP of DF-AF selection MIMO relaying over keyhole channels.Theorem 8.The outage probability and SEP for DF-AF selection MIMO relaying over keyhole channels can be bounded by and respectively, where and Proof.The instantaneous total SNR at the destination is Meanwhile, we have Subsequently, applying independency as well as Proposition 3 in Appendix, we arrive at (46) and (47).Combining (48), (49) in addition with ( 27) and ( 28), ( 44) and ( 45) can be derived.
It can be shown that f sep Ψ γ 1 (s) → 0 when γ 1 is sufficiently large.Then (46) and (47) become Remark: We consider the MIMO relay channels when the source-destination, the source-relay, and the relaydestination channels all incur keyhole effect in this paper.This scenario is more complicated than the cases that only one or two channels incur keyhole effect.Moreover, we consider Nakagami-m fading environments.Since Nakagami-m fading is the generalization of Rayleigh fading, it is a more complex channel fading model than Rayleigh fading.Based on the above two reasons, we can explain why the derived results appear extremely complex.The complexity mainly exists in computing the MGFs (Lemma 1) and in the computation of the inverse Laplace transform (Eqns.( 27), (28)).Regarding Lemma 1, the computations of the parameters ̺ w,p,j and ϑ w,q,l EAI European Alliance for Innovation are somewhat complicated.However, there are numerical methods to evaluate the partial derivatives [51], and an efficient method can be found in [52].In Eq. ( 23), the Whittaker function W1−j−l 2 , j−l 2 − 1 σ w sλ w,p ε w,q appears complex.The computation complexity of such special function has been investigated in [53].In Section 4, we use the "WhittakerW" function in the Matlab for the computation, and the execution time is acceptable.With respect to the inverse Laplace transform, there are a lot of numerical computation algorithms [54].In [54], the comparisons of the accuracy and the computation time were performed.In the evaluations, we use the method proposed in [55].This method can achieve better efficiency and accuracy by accelerating the convergence of the Fourier series obtained from the inversion integral using the trapezoidal rule.
Finally, we compare the three protocols as follows.
Lemma 2. In terms of the outage probability and symbol error probability over keyhole channels, the DF-AF selection MIMO relaying protocol is better than the AF MIMO relaying and the DF MIMO relaying.+1 .Comparing ( 32), ( 38) with ( 48), we have γ > γ df and γ > γ af .Thus, DF-AF selection MIMO relaying has lower outage probability and symbol error probability.

Numerical results
In this section, computer simulations are conducted to verify the accuracy of our analytical results.We show Monte-Carlo simulation results and compare them with our analysis.
Figure 3 shows the outage probability of DF-AF selection MIMO relaying, AF MIMO relaying and DF MIMO relaying with different numbers of antennas at the destination, n d .We assume equal power allocation, i.e., P 0 = P 1 = 1/2P.We can notice that DF-AF selection MIMO relaying has the same outage performance as DF MIMO relaying and has better outage performance than AF MIMO relaying.It can also be noted that the number of antennas at the destination has a strong impact of the performance enhancement, since the diversity order is n d n s + n r min{n d , n s }.Observe that simulation curves match in high accuracy with analytical ones.
Figure 4 plots the diversity order with respect to n s and n d .In the simulations, we set n r = 2.We can observe the relations in the figure.For example, the diversity order is 2n d + 4 for n d ≥ 2 when n s = 2.To further demonstrate the advantages of the DF-AF selection MIMO relay scheme, we show the outage performance of the three schemes when MRC is used in Figure 5.In the simulations, we utilize different values of power allocation, θ = P 0 P .From the figure, we can see that the DF-AF selection MIMO relay scheme has better outage performance than DF MIMO relaying and AF MIMO relaying.It can also be observed that the outage probability first decreases and then increases with the increase of θ.It is because that when θ is small, the decoding at the relay fails with high probability,  i.e., AF protocol will be applied, and γ = γ 0 + +1 .With the increase of θ, |γ 1 − γ 2 |, i.e., the difference between γ 1 and γ 2 decreases and then +1 increases, so the outage probability decreases.Once θ is larger than a certain value, the relay could correctly decode the source message with high probability.Then DF protocol will be used and γ = γ 0 + γ 2 .In this case, γ 2 with decreases with the increase of θ, then the outage probability will increase.

Keyhole scenario
The outage probability and SEP of different MIMO relaying schemes over keyhole channels are evaluated.We set P 0 = P 1 , c 0 = c 1 = c 2 = 1 and employ BPSK modulation in the simulations.The source and the relay use the same OSTBCs.We consider two kinds of OSTBC schemes: Alamouti code and G 3 [56].
Figure 6 and Figure 7 plot the outage probability performance and SEP performance of keyhole channels with DF-AF selection MIMO relaying, AF MIMO relaying and DF MIMO relaying when Alamouti code is used at the source and the relay (n s = n r = 2, R = 1), respectively.To compare the impact of the number of antennas at the destination (n d ), the outage probabilities and SEPs at different the numbers of antennas at the destination are presented.
Figure 8 and Figure 9 illustrate the outage probability performance and SEP performance of keyhole channels with DF-AF selection MIMO relaying, AF MIMO relaying and DF MIMO relaying when G 3 is used at the source and the relay (n s = n r = 3, R = 1/2), respectively.(ii) Regarding outage probability performance and SEP performance over keyhole channels, the DF-AF selection MIMO relaying is better than the DF MIMO relaying which is better than the AF MIMO relaying.
(iii) The OSTBC scheme used at the source and relay as well as the number of antennas at the destination plays an important role in the performance evaluations.
Proposition 3. Let Y = cX, where X is random variable and c 0 is constant.Then Ψ Y (s) = Ψ X (cs). Proof.

Figure 1 .
Figure 1.MIMO relay channel Let P = P 0 + P 1 , P 0 = θP.Define SNR = P N 0 .The diversity order d = − lim SNR→∞ log P out log SNR can be give by the following theorem.

Theorem 4 .
The diversity order of AF MIMO relaying is d af = n d n s + min{n s , n d }n r c 0 , c 1 and EAI European Alliance for Innovation 6 EAI Endorsed Transactions on Wireless Spectrum 05-07 2014 | Volume 01 | Issue 2 | e3

Figure 2 .
Figure 2. Keyhole effect in MIMO channel

Figure 3 .
Figure 3. Outage performance of DF-AF selection MIMO relaying, DF MIMO relaying, and AF MIMO relaying with different values of n d .

Figure 4 .
Figure 4. Diversity performance of DF-AF selection MIMO relaying

Figure 5 .
Figure 5. Outage performance of DF-AF selection MIMO relaying, DF MIMO relaying, and AF MIMO relaying with n d = 2 and P = 10dB.

Proposition 2 .F
C.d.f. of ζ defined in Proposition 1 is given by

Table 2 .
Notations Cooperative MIMO Relaying with Orthogonal Space-Time Block Codes in Wireless Channels with and without Keyholes Theorem 7. The outage probability and SEP for the AF MIMO relaying over keyhole channels can be bounded by European Alliance for Innovation 8 EAI Endorsed Transactions on Wireless Spectrum 05-07 2014 | Volume 01 | Issue 2 | e3 Volume 01 | Issue 2 | e3 Cooperative MIMO Relaying with Orthogonal Space-Time Block Codes in Wireless Channels with and without Keyholes For the keyhole-free scenario, we investigate the outage probability of the DF-AF selection MIMO relaying.The closed-form outage probability and diversity order are derived.For comparison purpose, we also obtain the outage and diversity of the DF MIMO relaying and the AF MIMO relaying.For the scenario that the keyholes exist, the outage probability and symbol error probability of OSTBC over MIMO relay channels with keyholes are analyzed.Exact outage probability and symbol error probability are obtained for DF MIMO relaying.With respect to AF MIMO relaying and DF-AF selection MIMO relaying, lower and upper bounds are derived.In addition, we proved that DF-AF selection MIMO relaying protocol has the best performance in both scenarios.Numerical results verify our proposed analysis.