Research Article
Distributed Cloud Monitoring Platform Based on Log In-Sight
@INPROCEEDINGS{10.1007/978-3-030-48513-9_6, author={E. Haihong and Yuanxing Chen and Meina Song and Meijie Sun}, title={Distributed Cloud Monitoring Platform Based on Log In-Sight}, proceedings={Cloud Computing, Smart Grid and Innovative Frontiers in Telecommunications. 9th EAI International Conference, CloudComp 2019, and 4th EAI International Conference, SmartGIFT 2019, Beijing, China, December 4-5, 2019, and December 21-22, 2019}, proceedings_a={CLOUDCOMP}, year={2020}, month={6}, keywords={Log insight Distributed cloud monitoring Log analysis}, doi={10.1007/978-3-030-48513-9_6} }- E. Haihong
Yuanxing Chen
Meina Song
Meijie Sun
Year: 2020
Distributed Cloud Monitoring Platform Based on Log In-Sight
CLOUDCOMP
Springer
DOI: 10.1007/978-3-030-48513-9_6
Abstract
Log management plays an essential role in identifying problems and troubleshoot problems in a distributed system. However, when we conducted log analysis on big data cluster, Kubernetes cluster and Ai capability cluster, we found it was difficult to find a Distributed cloud monitoring platform that met our requirements. So, we propose a Distributed cloud monitoring platform based on log insight, which can be used to achieve unified log insight of big data clusters, K8s clusters, and Ai capability clusters. At the same time, through this system, Developers can intuitively monitor and analyze the business system data and cluster operation monitoring data. Once there is a problem in the log, it will immediately alert, locate, display, and track the message. This system is helpful to improve the readability of log information to administrators, In the process of data collection, Filebeat and Metricbeat will be combined to collect data, therefore, the system can not only collect ordinary log data but also support to collect the indicator data of each famous mature system (Such as operating system, Memcached, Mysql, Docker, Kafka, etc.). Besides, the system will monitor and manage the status of cluster nodes through BeatWatcher. Finally, we develop the system and verify its feasibility and performance by simulation.
