Cloud Computing, Smart Grid and Innovative Frontiers in Telecommunications. 9th EAI International Conference, CloudComp 2019, and 4th EAI International Conference, SmartGIFT 2019, Beijing, China, December 4-5, 2019, and December 21-22, 2019

Research Article

Distributed Cloud Monitoring Platform Based on Log In-Sight

Download
63 downloads
  • @INPROCEEDINGS{10.1007/978-3-030-48513-9_6,
        author={E. Haihong and Yuanxing Chen and Meina Song and Meijie Sun},
        title={Distributed Cloud Monitoring Platform Based on Log In-Sight},
        proceedings={Cloud Computing, Smart Grid and Innovative Frontiers in Telecommunications. 9th EAI International Conference, CloudComp 2019, and 4th EAI International Conference, SmartGIFT 2019, Beijing, China, December 4-5, 2019, and December 21-22, 2019},
        proceedings_a={CLOUDCOMP},
        year={2020},
        month={6},
        keywords={Log insight Distributed cloud monitoring Log analysis},
        doi={10.1007/978-3-030-48513-9_6}
    }
    
  • E. Haihong
    Yuanxing Chen
    Meina Song
    Meijie Sun
    Year: 2020
    Distributed Cloud Monitoring Platform Based on Log In-Sight
    CLOUDCOMP
    Springer
    DOI: 10.1007/978-3-030-48513-9_6
E. Haihong1,*, Yuanxing Chen1,*, Meina Song1,*, Meijie Sun1,*
  • 1: Beijing University of Posts and Telecommunications
*Contact email: ehaihong@bupt.edu.cn, chenyuanxing_bupt@foxmail.com, mnsong@bupt.edu.cn, sun_magine@163.com

Abstract

Log management plays an essential role in identifying problems and troubleshoot problems in a distributed system. However, when we conducted log analysis on big data cluster, Kubernetes cluster and Ai capability cluster, we found it was difficult to find a Distributed cloud monitoring platform that met our requirements. So, we propose a Distributed cloud monitoring platform based on log insight, which can be used to achieve unified log insight of big data clusters, K8s clusters, and Ai capability clusters. At the same time, through this system, Developers can intuitively monitor and analyze the business system data and cluster operation monitoring data. Once there is a problem in the log, it will immediately alert, locate, display, and track the message. This system is helpful to improve the readability of log information to administrators, In the process of data collection, Filebeat and Metricbeat will be combined to collect data, therefore, the system can not only collect ordinary log data but also support to collect the indicator data of each famous mature system (Such as operating system, Memcached, Mysql, Docker, Kafka, etc.). Besides, the system will monitor and manage the status of cluster nodes through BeatWatcher. Finally, we develop the system and verify its feasibility and performance by simulation.