12th EAI International Conference on Testbeds and Research Infrastructures for the Development of Networks & Communities

Research Article

Spark Platform Based Video Transcoding

Download6 downloads
  • @INPROCEEDINGS{10.4108/eai.28-9-2017.2273775,
        author={Yunyu Liu and Jinpeng Yuan},
        title={Spark Platform Based Video Transcoding},
        proceedings={12th EAI International Conference on Testbeds and Research Infrastructures for the Development of Networks \& Communities},
        publisher={EAI},
        proceedings_a={TRIDENTCOM},
        year={2018},
        month={1},
        keywords={spark; ffmpeg; mapreduce; video transcoding},
        doi={10.4108/eai.28-9-2017.2273775}
    }
    
  • Yunyu Liu
    Jinpeng Yuan
    Year: 2018
    Spark Platform Based Video Transcoding
    TRIDENTCOM
    EAI
    DOI: 10.4108/eai.28-9-2017.2273775
Yunyu Liu1,*, Jinpeng Yuan1
  • 1: Qiannan Normal University for Nationalities
*Contact email: 18285421821@139.com

Abstract

The HTML5 based videos play an important role in promoting the communication on national culture with the rapid development of the mobile internet. However, considering that the HTML5 based videos support Theora, H.264 and MPEG4 video coding formats only and there are various existing video formats on national culture, it is needed to conduct fast conversion on video files so as to adapt to HTML5 video labels. Therefore, a Spark platform based transcoding system is proposed in this article. The HDFS is adopted for storage, and the RDD (Resilient Distributed Dataset) and FFMPEG of Spark are utilized for distributed transcoding. It conducts detailed discussion on segmen-tation strategy for the distributed storage of videos, and makes comparisons on the thought of the MapReduce and that of the RDD. In addition, it proposes the RDD programming framework based distributed transcoding scheme. Ac-cording to the comparisons on time consumed for transcoding between the MapReduce framework and the Spark framework with the same size of file block and cluster, compared with the MapReduce transcoding, the time used for transcoding of the Spark framework can be reduced by 25%