Research Article
A Method of Data Integrity Check and Repair in Big Data Storage Platform
3 downloads
@INPROCEEDINGS{10.1007/978-3-030-57115-3_15, author={Jiaxin Li and Yun Liu and Zhenjiang Zhang and Han-Chieh Chao}, title={A Method of Data Integrity Check and Repair in Big Data Storage Platform}, proceedings={Bio-inspired Information and Communication Technologies. 12th EAI International Conference, BICT 2020, Shanghai, China, July 7-8, 2020, Proceedings}, proceedings_a={BICT}, year={2020}, month={8}, keywords={HDFS Data integrity checking Data recovery Hash function}, doi={10.1007/978-3-030-57115-3_15} }- Jiaxin Li
Yun Liu
Zhenjiang Zhang
Han-Chieh Chao
Year: 2020
A Method of Data Integrity Check and Repair in Big Data Storage Platform
BICT
Springer
DOI: 10.1007/978-3-030-57115-3_15
Abstract
In the big data storage platform, in order to ensure the security of user data, it is necessary to perform cyclic verification on the stored data and repair the damaged data in time. Considering the problems of low verification efficiency, low check frequency and low calibration accuracy of HDFS data integrity check, this paper proposed a new HDFS storage platform security check and repair scheme. The process can effectively reduce the amount of calculation and communication overhead, and can support the dynamic operation of the data. Experiments show that this method has certain advantages in security, scalability and flexibility.
Copyright © 2020–2025 ICST
