Research Article
Design of Enterprise Economic Dynamic Management System Based on Spark Technology
@INPROCEEDINGS{10.1007/978-3-031-18123-8_21, author={Lu Zhang and Yipin Yan}, title={Design of Enterprise Economic Dynamic Management System Based on Spark Technology}, proceedings={Multimedia Technology and Enhanced Learning. 4th EAI International Conference, ICMTEL 2022, Virtual Event, April 15-16, 2022, Proceedings}, proceedings_a={ICMTEL}, year={2022}, month={10}, keywords={Spark technology Enterprise economy Dynamic management Data warehouse Spark Join operator}, doi={10.1007/978-3-031-18123-8_21} }- Lu Zhang
Yipin Yan
Year: 2022
Design of Enterprise Economic Dynamic Management System Based on Spark Technology
ICMTEL
Springer
DOI: 10.1007/978-3-031-18123-8_21
Abstract
Aiming at the problem that the currently used dynamic management system based on Hadoop and B/S architecture is affected by the slow data mining rate, resulting in low efficiency of data dynamic management, a design of enterprise economic dynamic management system based on Spark technology is proposed. Deploy the physical architecture of the Spark-based economic dynamic management system, and build a data warehouse in this architecture to facilitate users to quickly view data in real time. The B/S (browser/server) model is adopted to design data collection modules, business service modules and performance modules to meet the needs of big data analysis and decision-making. When using Spark technology to dynamically adjust the difference data in the database, the rule base needs to be updated in time to convert the automatic conversion system to a detection system. Use the optimization algorithm of Spark Join operator to optimize the entire connection operation, filter out the project data without specific categories in the bank flow data, reduce the data entering the shuffle stage, and design a dynamic management process. It can be seen from the test results that the system has a maximum management efficiency of 92% in a safe environment. In a non-interference environment, the highest management efficiency is 0.95, which has an efficient management effect.
