Information Outage Probability and Constrained Capacity of Moderate-Length Codes over AWGN Channels

We study the information outage probability (IOP) and constrained capacity of moderate-length codes over AWGN channels based on M-ary phase-shift keying signals. The IOP provides an important benchmark for performance evaluation of moderate-length codes. We analytically compute the IOP and compare it with numerical simulations using the DVB-S2 error-correcting code with numerous code rates employed in Protected Tactical Waveform (PTW). Numerical results confirm the tightness of the analytical results. Received on 03 October 2021; accepted on 01 November 2021; published on 03 November 2021


Introduction
From the information theoretic measure, channel capacity provides a fundamental understanding of the performance on a signaling type over a channel. It addresses the problem of efficient data transmission over the channel. Channel capacity is closely related to mutual information, which provides a reduction in uncertainty about the channel input given the knowledge of the channel output. Wide classes of channels have been studied based on probability theory such as ergodic theory or laws of large numbers. In order to establish the channel capacity or ergodic channel capacity result, the code block length is allowed to grow without bound. Channel capacity is related to the average mutual information only in the limit of infinite-length codewords. If the coding length is required to be infinite for the average mutual information to converge, then one might infer that the real value of interest is not the average ★ A part of this work was presented at IEEE Military Communications Conference, Los Angeles, CA, USA, October 2018 [1]. * Corresponding author. Email: nghi.tran@uakron.edu mutual information, but the mutual information itself. Also, recent results on channel capacity for a general class of channels; the average mutual information is not sufficient for that purpose [2]. This observation motivates us to study mutual information itself, which then provides insight into the performance of finitelength codewords.
In the early development of information theory, mutual information between the channel input and output is treated as a random variable [2][3][4]. The failure probability or the information outage probability (IOP), is defined as the probability of this random variable drops below the code rate R [5]. The IOP is simply referred as the outage probability in this paper. A formal approach of IOP was presented in Feinstein's Lemma [4], which states that a code exists with maximal codeword error probability that is incrementally higher than the IOP. Feinstein's Lemma suggests that the IOP could be used as a benchmark for the codeword error probability. In previous works, authors focused IOP on fading channel [6,7]. In this model, each codeword is divided into K timeslots, each of duration T , and each timeslot is transmitted through a fading channel. The signal to noise ratio (SNR) is assumed to be constant in each timeslot but may vary from one timeslot to the next timeslot. The overall mutual information is found by averaging across K timeslots [8][9][10]. In general, IOP does not provide a lower decoding bound [8,11]. It bounds on the maximum probability of error, not the average probability of error. It provides a good performance benchmark of the achievable decoding error rates. For this reason, the performance bound simply means the performance benchmark in this paper.
In the regime of finite-length codes, the IOP also appeared in the literature in the context of information spectrum methods [2,8,[12][13][14][15]. The IOP is simply the cumulative distribution function (cdf) of the mutual information rate random variable, evaluated at code rate R. The main utility of the cdf is that it can be analytically obtained by the Gaussian Qfunction, instead of simulating the actual code. It, however, provides a performance guidance for the code designers.
In the regime of moderate-length codes where the number of symbols per codeword is n > 1000, the IOP for QPSK, LTE turbo code is discussed in [11]. A numerical method for BPSK, LTE turbo code is discussed in [16]. As shown in [11,16], in order to achieve maximum AWGN capacity, the input signal is assumed to follow a Gaussian distribution function. Results shown in [11,16] indicate that the IOP is about 0.7 dB and 1.2 dB better than the block error rate (BLER), respectively. Motivated by these results, we study the performance of Protected Tactical Waveform (PTW) [17,18], which employs M-PSK signals, using Second Generation Digital Video Broadcasting Satellite (DVB-S2) forward error correcting (FEC) code with numerous code rates, and a fixed length of 16, 200 coded bits. DVB-S2 FEC is based on the concatenation of BCH (Bose-Chaudhuri-Hocquengham) and LDPC (Low Density Parity Check) codes. PTW has been adopted by the Air Force to use over the commercial and Military Satellite Communication (MILSATCOM) systems. PTW supports very high data rates while providing secured and anti-jam protection, as well as low probability of intercept and detection (LPI/LPD) [19] capability.
The IOP results obtained by previous studies [11,16] are satisfactory only for the codes which operate in the region of low to moderate SNR where the maximum capacity or Shannon capacity matches the constrained capacity. These results rely on Gaussian distribution assumption of the input signal. However, the Gaussian input assumption results in loose performance bound for high modulation orders where FEC operates in high region of SNR [20]. The main contribution of the paper is to provide a general solution for the IOP for a wide range of SNR, which includes high orders of modulation and high code rates. We do not rely on the Gaussian distribution of the input signal. A uniform distribution is realistic and used in the paper. To best of our knowledge, this paper is the first work that examines the IOP by relaxing the Gaussian input assumption. We show that the IOP provides a very tight performance bound, specifically, the theoretical performance benchmark for PTW, a future Military Standard waveform for MILSATCOM. In the preliminary work reported in [1], we presented our approximation approach to find the IOP and constrained capacity of moderate-length codes. In this work, we provide more thorough investigation and comparison between the proposal approach to results achieved by Gaussian signaling. In the simulation section we show the Gaussian assumption results in very loose performance bound for codes that operate in the region of high SNR; it exceeds 3.6 dB for 8-PSK with code rate 8/9, while our approach results in a very tight bound, less than 0.3 dB. The results of this paper can be extended to study the performance for a wide class of codes.
The outline of the paper is as follows: In Section II, a review of mutual information, average mutual information, and mutual information rate. The calculation of IOP is based on additive Gaussian noise (AWGN) channels. There are two parts: Part I when input M-PSK signal is assumed to follow a Gaussian distribution; Part II, we relax the Gaussian distribution of the input signal and a more realistic uniform distribution is used. In part II, we show that the received signal is a Gaussian mixture. Therefore, maximum capacity is not achieved in the region of high SNR. In addition, for high orders of modulation, e.g., QPSK or higher, there is no closed form solution for the average mutual information and the information rate random variable. The calculation of IOP is based on the constrained capacity of the M-PSK signals. We present approximation and numerical integration methods to compute the constrained capacity. Details of these methods are discussed in sections II and III of the papers. Finally, simulation results using PTW which employs DVB-S2 FEC code with numerous code rates and the IOP obtained in Section II are presented in Section 2, followed by some concluding remarks in Section 3.
Notations: We use capital letters to denote random variables, small letters to indicate sample values, bold letters to indicate vectors of sample values. To simplify the analysis, complex baseband signaling is considered. In addition, codeword error rate and block error rate are used interchangeably.

System and Channel Model
We consider a one-to-one communication link. At the receiver, the discrete time output signal of the additive 2 EAI Endorsed Transactions on Industrial Networks and Intelligent Systems 09 2021 -11 2021 | Volume 8 | Issue 29 | e5 white Gaussian channel noise is given by where X denotes the Gray-coded M-PSK transmitted symbols, and N denotes the additive white Gaussian noise (AWGN). The probability density function (pdf) of the complex AWGN is given by where N 0 denotes the noise power density. The mutual information between samples x and y is given by where log(·) denotes the natural logarithmic function, f Y |X (y|x) denotes the conditional pdf of Y given X; it is given by The mutual information i(x; y) is a random variable with unit of nats/channel use. The average mutual information is given by where E{·} denotes the expectation operator. The average mutual information (5) can also be expressed as The above expression suggests that the average mutual information can be estimated based on the mutual information of the input and output vectors of length n [8], i.e., where n denotes the number of symbols per codeword. The expression on the right-hand side (RHS) of (7) is also known as the mutual information rate. Mutual information rate is a random variable.

Part I: Gaussian Distributed Input
Suppose the input random variable X is a zero-mean, complex Gaussian random variable, the pdf of X is given by where E s denotes the symbol energy. Since both signal X and noise N are zero-mean complex Gaussian random variables, the output signal Y is also a zero-mean complex Gaussian random variable, and the pdf of Y is given by Substituting (4) and (9) into (3), the mutual information is Substituting (10) into (5), the average mutual information is where the signal to noise ratio (SNR) is given by The channel capacity or the Shannon capacity for the AWGN is given by [3] C 0 (γ) = sup As shown in (11) and (13), for AWGN channel, the maximum capacity C 0 is obtained when the pdf of X follows a Gaussian distribution. Given that (10), (11), and (13), the mutual information rate random variable shown in (7) can be shown to have the same distribution as random variable Z n ; it is given by where where n denotes the number of symbols per codeword.
Since Y and N are zero-mean Gaussian random variables, |Y | 2 and |N | 2 are chi-square distributed random variables. For two degrees of freedom (complex data), chi-square distribution becomes exponential distribution. Furthermore, for n = 1, the difference of two exponentially distributed random variables is a zero-mean Laplacian random variable with variance [8,21] Again, γ is the SNR, given by (12). Using Central Limit Theorem, for large n (n > 100), W n is approximated by 3 EAI Endorsed Transactions on Industrial Networks and Intelligent Systems 09 2021 -11 2021 | Volume 8 | Issue 29 | e5 a Gaussian random variable. The distribution of the Z n can be approximated as .
Proposition 1. For infinite length codeword, the mutual information rate random variable Z n is the Shannon capacity.
Proof. Taking the limit of the variance of the mutual information rate random variable Z n as the codeword length approaches infinity, one obtains The mutual information rate random variable converges to the Shannon capacity. This result is consistent with (11) and (13).
The cumulative distribution function of Gaussian random variable Z n is given by where Q(x) is defined as Define the code rate as where k denotes the number of information bits and m denotes the number of coded bits per codeword. Code rate R c is normally specified in communications systems.
The effective code rate in bits/symbol is given by The outage probability P o is the probability that the mutual information rate is less than the code rate R e . The outage probability for Z n is given by or 2γ/π(γ + 1)

Part II: Uniform Input Distribution
For M-PSK communication systems, Gaussian distribution assumption for the input signal is not realistic. A more realistic distribution for X is uniform distribution; the pdf of X is given as where The pdf of the output random variable Y is the convolution of (2) and (25); it is given by or The mean value of the output signal is and the output signal power is where The variance of the output signal is then given by  Substituting (4) and (28) into (3), the mutual information is Substitute (33) into (5), the average mutual information is given as where Due to the presence of a mixture of Gaussian components, for high order of modulation, QPSK or higher (M ≥ 4), there is no closed form expression for I Y X and the average mutual information [22][23][24]. The constrained capacity of M-PSK signal that has finite alphabet M, uniformly distributed, it is given by As discussed in Section 2.1, under Gaussian input assumption, the outage probability is governed by the channel capacity. For uniform distribution assumption of the input signal, the outage probability is governed by the constrained capacity. For M-PSK signal, no closed form expression of the constrained capacity. An approximation, a numerical, and a Monte-Carlo simulation solution of the constrained capacity will be used.

Method 1: Approximation Constrained Capacity for M-PSK Signals
An exponential approximation that reduces the computational burden is given as [25] C con (γ) ≈ log 2 M where γ denotes the SNR, a i , b i , L are curve fitting parameters. Approximation parameters for QPSK and 8-PSK are summa-rized in Table 1. Fig. 2 depicts the approximate of the constrained capacity for QPSK and 8-PSK modulations and the Shannon capacity.

Method 2: Numerical Integration Constrained Capacity for M-PSK Signals
To evaluate (35) via numerical integration, make a change of variables where Y R and Y I denote the real and imaginary parts of sample y, respectively; and x j,R and x j,I denote the real and imaginary parts of sample x j , respectively.

Substituting (38) into (35), one obtains
Using the Gauss-Hermite quadrature [26] to numerically integrate (39), one obtains where s i and t j are roots of the Hermite polynomial and v i and w j are associated weights. Substituting (41) into (34), one obtains Follow a similar development to that described in Section 2.1 of the paper, the distribution of the mutual information rate random variable Z n between the input and output vectors of length n can be expressed as where Define For a single ring constellation of M-PSK signals, we have Substituting (45) and (47) into (44), after some algebra, one obtains Substituting (48) into (43), one obtains Define where Since Y k is a zero-mean Gaussian mixture, Y ′ k is also a zero mean Gaussian mixture, variance equals γ + 1, while N ′ k is a normalized Gaussian random variable, zero mean, variance equals 1. Therefore, W n is also a zero-mean random variable.
Given (5), (36), (43)-(49), in the mean of (49), one obtains Since |β| ≤ 1, we have Substituting (50) and (52) into (49), one obtains Again, C con is obtained by either approximation or numerically methods as given either by (37) and (42), respectively, and W n is given by (50). The mean of the mutual information rate random variable is governed by the constrained capacity. Again, similar to part I, for n = 1 and in the limit where Y follows a Gaussian distribution, W is a Laplacian random variable, with variance depends on γ. When Y is a Gaussian mixture of M Gaussian distributions, the variance of W is M times the variance of a single Gaussian distribution, i.e., For large n, (n > 100), using Central Limit Theorem, W n follows a Gaussian distribution. The mutual information rate random variable Z n shown in (44) can be expressed as Z n ∼ N C con (γ), 2Mγ n(γ + 1) .
Comparing this result against part I, although the distribution of the mutual information rate random variable Z n is Gaussian; however, the mean shifts toward the constrained capacity and the variance is M times larger.

Proposition 2.
For infinite length codeword, the mutual information rate random variable Z n is the constrained capacity.
Proof. Similar to Part I, taking the limit of the variance of the mutual information rate random variable Z n as the codeword length approaches infinity, one obtains lim n→∞ V ar(Z n ) = lim n→∞ 2Mγ n(γ + 1) = 0.
Thus, the mutual information rate random variable converges to the constrained capacity.
The outage probability P o is the probability that the mutual information rate is less than the code rate R e . The outage probability for Z n is given by or Again, R e is given by (22). For M-PSK signals, the IOP shown in (59) is applicable for the entire SNR region. Equation (59) provides the performance benchmark for moderate-length codes. Numerical simulation of the DVB-S2 FEC with various code rates and the IOP are discussed in the simulation section of the paper. As shown above, the mean of the mutual information rate random variable Z n is a function of constrained capacity. The performance of the outage probability is sensitive to the expected capacity, the degree in which the approximation, numerical integration, and Monte-Carlo simulation methods converge to the expected capacity. We will discuss the performance of the outage probability and numerical results in Section 3 of the paper.

Simulation Results
Computer simulations for Quadrature Phase Shift Keying (QPSK) and 8-Phase Shift Keying (8-PSK), using DVB-S2 FEC with a fixed length of 16, 200 coded bits, code rates R c of 1 / 4 , 1 / 3 , 1 / 2 , 2 / 3 , 3 / 4 and 8 / 9 are considered. Details information of the DVB-S2 FEC and the constellations of QPSK and 8-PSK are defined in [27]. Fig. 4 depicts the outage probability P o using numerical integration, approximation, Gaussian assumption, and simulated BLER for QPSK with low code rate R c = 1 / 4 , 1 / 3 . Fig. 5 depicts the outage probability P o using numerical integration, approximation, Gaussian assumption, and simulated BLER for QPSK with moderate code rate Fig. 6 depicts the outage probability P o using numerical integration, approximation, Gaussian assumption, and simulated BLER for 8-PSK with high code rate As shown in Figs • are tight for the numerical constrained capacity method. An interesting observation is for 8-PSK, rate 8/9, the outage probability curve crosses the simulated curve. In this SNR region, the constrained capacity is slightly underestimated; it shifts the outage probability curve to the right and causing its slope more gradual, not as steep as the simulated curve; • are also tight for the approximate constrained capacity method; with the exception of 8-PSK rate 3/4 where the numerical method is tighter for about 0.5 dB. In this case, the constrained capacity is slightly overestimated.  Table 2. Performance bound in dB for the outage probability P o , using numerical integration, approximation, and Gaussian input distribution assumption at BLER = 10 −4 .
capacity methods provide the tight bound for the outage probability.

Conclusion
Feinstein's lemma for the IOP is a practical approach and an important benchmark of achievable rate to predict the performance of moderate length codes. In this paper, we presented an approach to compute the IOP for M-PSK signals using DVB-S2 FEC with a fixed length of 16, 200 coded bits with numerous code rates for AWGN channel. Computer simulation, using PTW waveform which employs QPSK and 8-PSK signaling, DVB-S2 FEC with a fixed length of 16, 200 coded bits with numerous code rates were presented to compare the BLER against the outage probability under Gaussian and uniform distributions of the input signals. With the Gaussian inputs distribution assumption, the IOP results in the loosest bound. The bound exists 3.5 dB for 8-PSK, rate 8/9. When the code is operating in the high SNR region Shannon capacity exceeds the constrained capacity. Therefore, for high order of modulation and high code rate, one should not assume Gaussian input distribution. The bound is tight for the numerical constrained method. The performance bound in the range of −0.27 to 0.78 dB for BLER of 10 −4 . For 8-PSK, rate 8/9, the outage probability curve intersects with the simulated curve. In this region, the numerical constrained capacity is slightly underestimated; it shifts the outage probability curve to the right and causing its slope to be more gradual than the simulated curve. The bound is also tight and consistent for the approximate constrained capacity method. However, the bound peaks for 8-PSK, rate 3/4. In this case, the constrained capacity is slightly overestimated; it causes the outage proba-bility curve shifts to the left, which result in a performance bound of 0.96 dB at BLER of 10 −4 . For the approximation constrained method, the performance bound is in the range of 0.18 to 0.96 dB for BLER of 10 −4 . An interesting observation is that one can use the simulated curve to benchmark the rate of convergence for the constrained capacity. Comparing all the methods, for QPSK and 8-PSK with all the 8 EAI Endorsed Transactions on Industrial Networks and Intelligent Systems 09 2021 -11 2021 | Volume 8 | Issue 29 | e5 code rates under consideration, both the approximation and numerical methods for constrained capacity yield good predictor for the performance bound of moderatelength codes. The IOP technique presented in the paper can be extended to predict the theoretical benchmark for a wide class of moderate-length codes.