Research Article
Spark Platform Based Video Transcoding
@INPROCEEDINGS{10.4108/eai.28-9-2017.2273775, author={Yunyu Liu and Jinpeng Yuan}, title={Spark Platform Based Video Transcoding}, proceedings={12th EAI International Conference on Testbeds and Research Infrastructures for the Development of Networks \& Communities}, publisher={EAI}, proceedings_a={TRIDENTCOM}, year={2018}, month={1}, keywords={spark; ffmpeg; mapreduce; video transcoding}, doi={10.4108/eai.28-9-2017.2273775} }- Yunyu Liu
Jinpeng Yuan
Year: 2018
Spark Platform Based Video Transcoding
TRIDENTCOM
EAI
DOI: 10.4108/eai.28-9-2017.2273775
Abstract
The HTML5 based videos play an important role in promoting the communication on national culture with the rapid development of the mobile internet. However, considering that the HTML5 based videos support Theora, H.264 and MPEG4 video coding formats only and there are various existing video formats on national culture, it is needed to conduct fast conversion on video files so as to adapt to HTML5 video labels. Therefore, a Spark platform based transcoding system is proposed in this article. The HDFS is adopted for storage, and the RDD (Resilient Distributed Dataset) and FFMPEG of Spark are utilized for distributed transcoding. It conducts detailed discussion on segmen-tation strategy for the distributed storage of videos, and makes comparisons on the thought of the MapReduce and that of the RDD. In addition, it proposes the RDD programming framework based distributed transcoding scheme. Ac-cording to the comparisons on time consumed for transcoding between the MapReduce framework and the Spark framework with the same size of file block and cluster, compared with the MapReduce transcoding, the time used for transcoding of the Spark framework can be reduced by 25%