Research Article
GPU-accelerated Large-Scale Non-negative Matrix Factorization Using Spark
@INPROCEEDINGS{10.1007/978-3-030-12981-1_13, author={Bing Tang and Linyao Kang and Yanmin Xia and Li Zhang}, title={GPU-accelerated Large-Scale Non-negative Matrix Factorization Using Spark}, proceedings={Collaborative Computing: Networking, Applications and Worksharing. 14th EAI International Conference, CollaborateCom 2018, Shanghai, China, December 1-3, 2018, Proceedings}, proceedings_a={COLLABORATECOM}, year={2019}, month={2}, keywords={Non-negative matrix factorization GPU CUDA Spark}, doi={10.1007/978-3-030-12981-1_13} }- Bing Tang
Linyao Kang
Yanmin Xia
Li Zhang
Year: 2019
GPU-accelerated Large-Scale Non-negative Matrix Factorization Using Spark
COLLABORATECOM
Springer
DOI: 10.1007/978-3-030-12981-1_13
Abstract
Non-negative matrix factorization (NMF) has been introduced as an efficient way to reduce the complexity of data compress and its ability of extracting highly-interpretable parts from data sets, and it has also been applied to various fields, such as recommendations, image analysis, and text clustering. However, as the size of the matrix increases, the processing speed of non-negative matrix factorization algorithm is very slow. To solve this problem, this paper proposes a parallel algorithm based on GPU for NMF in Spark platform, which makes full use of the advantages of in-memory computation mode and GPU Single-Instruction Multiple-data Streams mode. The new GPU-accelerated NMF on Spark platform is evaluated in a 4-nodes Spark heterogeneous cluster using Google Compute Engine by configuring each node a NVIDIA K80 GPU card, and experimental results indicate that it is competitive in terms of computational time against the existing solutions on a variety of matrix orders. It can achieve a high speed-up, and also can effectively deal with the non-negative decomposition of higher-order matrices, which greatly improves the computational efficiency.
