Effect of Packet Loss and Reorder on Quality of Audio Streaming

In this paper, Audio QoE assessment experiments were conducted by using network parameters such as packet loss and reorder. The network model was created by using three computers, which includes sender, receiver, and middle between Linux based router machines was configured. NetEm tool was used to disturb the on-going network traffic to real-time packet loss and reorder in the broadcasting environment. VLC player was used to broadcasting and recording audio from sender to receiver machines and audios were presented to users to listen, and assign ratings according to their level of satisfaction. The results show the user’s satisfaction level is decreased when packet loss and reorder level is increased in audio streams. The user accepted a certain level of network traffic disturbance, however, the increment of disturbance in network traffic damage the audio quality below the then acceptable level. This research work will help to cloud service providers to provide QoE to users according to singed in service level agreements.


Introduction
Nowadays audio streaming is commonly used around the world such as audio conference, online learning, and music listening from clouds and VOIP commercial usage for cheaper call abroad [1,2,3].Cloud computing provide resources on pay per uses as well free services depend on the cloud type [4,5,6].Social networks such as Facebook, WhatsApp, and Viber, etc. also provide VOIP calling facility to the user by using the Internet [7,8].Audio streaming is more popular recent years due to the online listing from clouds, this service provides by several originations such as SoundCloud, mymusiccloud, emusic, etc. and online radio stations (iheartradio), which broadcast music and provide access of several stations [9,10,11,12,13].
User likes to play high quality (HQ) music from online clouds and radio stations and organizations also provides these service in HQ to ensure user's satisfaction to continue to access their services.Service providers motive to provide quality of service (QoS) to their customers but slow networks, mobility of users cause packet loss and delay, which decrease the quality of experience of end users [14,15].Organizations take technical measures to improve QoS for end users, but they never considered QoE to assess user need and requirement by using audio streaming [16,17,18].
Quality of experience is used to conduct the user reviews, perception and observation the products and services [19,20].Laghari and connelly defined "QoE is a blueprint of all human subjective and objective quality needs and experiences arising from the interaction of a person with technology and with business entities in a particular context [21]."QoE is categorized into two approaches, one is subjective and other is objective [22,23].User reviews, web survey or questionnaire, and interviews are the ways of taking subjective QoE and QoS data and human physiological data is part of objective QoE [24,25,26].
Delivery of audio streaming service with HQ is challenging for service providers due to slow networks, mobility and distance of users from the cloud also add few delays [27,28,29].Organizations did not know the acceptable level of packet loss or reorder in audio streaming, so there is need to analyse the acceptable level of QoE of the end user's on packet loss and reorder factors during the accessing to audio services.The main influence of this paper is to analyse and discover the consequence of packet loss and reorder on audio streaming over the user QoE.
During the research, we conduct an experiment on NetEm by broadcasting audio clips from sender to receiver and during the broadcasting packet loss and reorder was added in streams.Audios were recorded for users to listen and assign ratings according to quality of streaming and acceptable level of packet loss and delay.This paper is organized into 5 sections and section based on the literature review.Section 3 provides design and experiment methodology and section 4 elaborates results and discussion.Finally, in section 5 we conclude the research work.

Literature Review
Mostly QoE was captured for video streaming services, voice over internet protocol (VOIP) and applications, but never considered for audio streaming of music clouds and radio stations [30,31,32].Some work found subjective and objective QoE assess of 3D mobile audio quality and audio archive evaluation [33,34].Mobile technologies and stereo systems used 2D audio streams for end users for listening, but this will cause damage of eardrum and user cannot listen for a long time [35,36].This regards Toosy et al. assess QoE 3D audio in mobile communication to perceive user experience and satisfaction compared to 2D audio [33].This result shows that 3D audio increases perceive the quality of audio as compared to other audio formats.
Alfayly et al. proposed a QoE based scheduling algorithm for downlink in VOIP, which was capable to increase the number of end users in per cell with QoE provision for VOIP applications [37].The proposed algorithm was applied in LTE-sim and its performance was compared with other LTE scheduling algorithms.The QoE driven scheduling algorithm differs by assigning priority to users by providing resource allocation for QoE necessities for VoIP Applications as compared to existing scheduling algorithms.The results show that the proposed QoE based scheduling algorithm improved downlink scheduling and increased cell size 75% compared modified least weighted delay first and 250% compared to Proportional Fair and Exponential/Proportional Fair.Wu et al. assess the QoE of three VOIP applications such as MSN Messenger, Google Talk, and Skype by applied playout buffer dimension algorithms [38].The experiment was conducted on the behaviour of applications how they adjust playout buffer sizes.Objective QoE metrics were used for data for collection from applications and results show that MSN and Google Talk do not control their own buffer size suitable way; on the other hand, Skype did not manage the buffer size at all.Effect of packet loss and delay was also measured in buffer size adjustment, but none of the application manages its buffer on the network loss or delays, and this will also be considered for service delivery with better QoE.Better QoE can be provided to end user by advancing their buffer dimension algorithms.
A methodology for enhancing the QoE of audio/video transmission on IP was proposed by Tasaka et al., which manages packet loss in video with error disguise/skipping frame of video and this uses adjustment of the relation of QoE between the temporal quality and spatial produced by two techniques [39].A scheme was adopted for switching among two techniques allowing to the proportion of slices errors of video hidden in a frame and name of the scheme was used as Switching between error Concealment and frame Skipping (SCS).Experiments were conducted using SCS with six contents.The cross model interaction was used between video and audio, the psychological scale was used for QoE assessed, which is more precise the mean opinion score (MOS).The results elaborate that QoE was improved by using SCS on simple frame skipping or error disguise by applying suitable threshold value on error disguise ratio, which based on the type of content, degree of video motion and video picture pattern.
Live broadcasting or on-demand services of audio/video utilize more bandwidth and cover more part of Internet traffic around the world.Nowadays, live or on-demand video streaming organization use HTTP adaptive streaming, this is suitable for the de-facto standard for streaming solution for audio/video.Karn et al. proposed a method to measure the bandwidth of the service receiver to attain high level of QoE in multi-user scenario [40].This method will guess the available bandwidth of the network, which depends on segment throughput and buffer status.The buffer model of the video is linked with three thresholds, two for operating and one for basic start-up thresholds.The performance of HTTP adaptive streaming was measured with NS-3 Simulator and results show that outputs reflect that the suggested model increases the QoE compared with previous conventional methods.
In past QoE of video streaming was measured by considering different factors such as packet loss, reordering, delay and compression however QoE of audio streaming was never measured by applying packet loss and reordering during the online play of music or audios.

QoE Assessment Platform Design and Experiment
Two audio streams were used for QoE assessment experiment, which were downloaded from different sources and having different content properties [41,42].Details of audio streams are given in Table 1.The private LAN network was designed to conduct an experiment, which was based on three computers.The sender and receiver machines were using Windows operating systems and middle router based on the GNU/Linux Centos was installed and configured for routing by using NetEm tool.The illustration of network topology is given in Figure 1.The audios were broadcasted from sender to receiver machine and network traffic was artificially disturbed by applying packet loss and reordering commands in NetEm tool.At the receiver end artificially manipulated audios were recorded for user listening purpose.

Figure 1. Network Model
The Absolute category rating (ACR) methodology was used for ratting purpose and Table 2 shows ACR rating scale and experimental methodology is shown in Fig. 2 [44].QoE study was collected in the laboratory of computer system engineering department of QUEST Nawabshah.Overall 61 participants were invited from computer system engineering and few of them from different departments.Most of the users belonged to undergraduate studies and the rest of them were master students.Subjects have age between 21 to 33 years, 34 of them were male and 27 were female students.A questionnaire was given to participants to give their profile data and assign ratings, according to the quality of audio as they perceive.For subjects, first original audios were played and then audio which contains packet loss and reorders were played from minor to the major rate of traffic manipulated, however, the subjects were oblivious of the packet reorder percentage in every audio.The speakers and headphones were used for common listening of audio for all subjects to assess quality properly.If the users observe that the audio quality is superior, then they give rating as brilliant, and if they are just satisfied then they rate audio as fair.If they have totally disagreed, the quality of the audio is irritating and rating as worse.During experiments, international telecommunication union ITU standard of mean opinion score (MOS)/ACR EAI Endorsed Transactions on Scalable Information Systems 10 2019 -01 2020 | Volume 7 | Issue 24 | e4 scale was followed design questionnaire for data collection, which is given in Table 2 [43].During the data analysis, 11 questionnaires of subjects were discarded because they provide the same ratings and did not understand how to assign ratings and required their response.

Figure 2. ACR experimental methodology
We estimate the MOS of assigned ratings of users by using Mean formula and results of quantitative data analysis of subjective QoE of experiments are given in the next section with details discussion.
Here, MOS is Mean Opinion Score ∑ represents the summation X represents scores N represents number of scores.

Results and Discussion
The VLC player (version 3.0.7.1) was used to broadcasting and recording of audios, which were artificially manipulated by using NetEm tool and File Viewer Lite (version 1.4) was used to extract codec information of audio clips [45,46].The purpose of using two different audios such as one is music without voice, and other is a voice without music and two network parameters to get more data of user's perception and satisfaction for future development and QoS provision.The motivation behind the using of packet loss and reordering is that in packet loss situation data is lost and which will never recover, so in this situation, users cannot understand during the listening of audios, which word is vanished from the stream.This situation is worse than packet reordering, where packets arrived on the destination without sequence and streams were mixed, but data was not lost.The results of packet loss in Table 3 and Figure 3 show that the user assigns high ratings to original audios as compared to audio with packet loss.The increment of packet loss effect more The Hobbit voice as compared to Music audio because The increment of packet loss effect more to The Hobbit voice as compared to Music audio because every packet of The Hobbit voice contains important information, so the if packet loss ratio is increased then ratings also decreased, which is the cause of low user perception.However, the increment of packet loss has less effect on Music audio because it contains the only melody no other voice information.Same way packet reordering commands were applied in NetEm tool to measure the impact of packet reordering on audio streams.In the first example, 5% of data packets (with a correlation of 10%) will be forwarded instantly others will be late by 10 ms. this reasons a certain number of the packets to become reordered [47,48,49] results show in Table 4 and Figure 4 that the number of packet reordering ratio was increased than MOS of user's decreased for both audio streams.There is a slight difference of ratings of Music and The Hobbit voice streams on packet reordering parameters because simple music stream was not affected as like voice stream and network disturbance made difficult to understand the voice in the clip.

Conclusion
In this paper, QoE of audio streams was measured by artificially disturbing network traffic by using NetEm tool to create real-time traffic scenario.Audios were broadcasted and recoded by using VLC player from sender to receiver, and users were invited to listen and assign ratings according to they receive the quality of audio.The results show that only 3% packet loss was fair for Music clip however, 2% is acceptable for the voice clip.If the packet loss was increased up to 5%, which is not acceptable for users and user MOS was also decreased.Packet reorder was also parameter was considered in experiment and results included in this paper showed that the maximum 15% 30% packet reordering level was acceptable for Music clip and The Hobbit voice have slightly less than fair level.If the packet reordering level was increased up to 20% 40%, which was not acceptable for users and they assigned ratings below the fair level.This research work provides a solution to audio music service provider's cloud if network traffic is disturbed certain level then audio quality also is disturbed, which is not acceptable for listening to online music from clouds.

Figure 3 .
Figure 3. Impact of Packet loss

Figure 4 .
Figure 4. Impact of Packet reorder

Table 1 .
Details of Audio Streams